2024-04-22T06:23:48.666 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-22T06:23:48.666 DEBUG:teuthology.run:Teuthology command: teuthology --name leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi --archive /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668133 --owner scheduled_leonidus@teuthology --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/with-quiesce 2-workunit/fs/trivial_sync}} --verbose -- /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668133/orig.config.yaml 2024-04-22T06:23:48.719 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-22T06:23:48.814 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668133 branch: wip-lusov-quiescer description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/with-quiesce 2-workunit/fs/trivial_sync}} email: null first_in_suite: false job_id: '7668133' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiescer ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 conf: client: client mount timeout: 600 client_shutdown_timeout: 120 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds oft prefetch dirfrags: true mds op complaint time: 180 mds verify scatter: true mds_dir_prefetch: true mds_sleep_rank_change: 5000000.0 mon client directed command retry: 5 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: client mds osd ms inject socket failures: 2500 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon client directed command retry: 5 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: client mds ms inject socket failures: 2500 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum sha1: 86f7587a5a09af35f0895e1a2d08527638fae697 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 86f7587a5a09af35f0895e1a2d08527638fae697 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiescer sha1: 86f7587a5a09af35f0895e1a2d08527638fae697 owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - mds.e - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - mds.f - osd.4 - osd.5 - osd.6 - osd.7 seed: 4362 sha1: 86f7587a5a09af35f0895e1a2d08527638fae697 sleep_before_teardown: 0 subset: 222/111111 suite: fs suite_branch: wip-lusov-quiescer suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 86f7587a5a09af35f0895e1a2d08527638fae697 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi044.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOkhDsKQt1CCnIug9tZrc1CemwV6jDC9sr/9o9GE8Hsi+FL16G3gngvPEsR3q4ADneLtalMZOZq8CmVR7TIRdeA= smithi070.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEcso0amEbD7LWkysbF+kOSM3IVibZdDd54bL/6XuJSZ7MZk97LGx/Xh4KeuLIRyxLqN83Xjbno3yb2y3nWBpwk= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - quiescer: null - workunit: clients: all: - fs/misc/trivial_sync.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-22_05:52:53 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-22T06:23:48.814 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa; will attempt to use it 2024-04-22T06:23:48.815 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks 2024-04-22T06:23:48.815 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-22T06:23:48.817 INFO:teuthology.task.internal:Checking packages... 2024-04-22T06:23:48.840 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '86f7587a5a09af35f0895e1a2d08527638fae697' 2024-04-22T06:23:48.840 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-22T06:23:48.840 INFO:teuthology.packaging:ref: None 2024-04-22T06:23:48.840 INFO:teuthology.packaging:tag: None 2024-04-22T06:23:48.840 INFO:teuthology.packaging:branch: wip-lusov-quiescer 2024-04-22T06:23:48.840 INFO:teuthology.packaging:sha1: 86f7587a5a09af35f0895e1a2d08527638fae697 2024-04-22T06:23:48.840 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiescer 2024-04-22T06:23:49.088 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3156.g86f7587a 2024-04-22T06:23:49.090 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-22T06:23:49.101 INFO:teuthology.task.internal:no buildpackages task found 2024-04-22T06:23:49.101 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-22T06:23:49.108 INFO:teuthology.task.internal:Saving configuration 2024-04-22T06:23:49.126 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-22T06:23:49.132 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-22T06:23:49.162 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi044.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668133', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-22 06:18:48.194634', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOkhDsKQt1CCnIug9tZrc1CemwV6jDC9sr/9o9GE8Hsi+FL16G3gngvPEsR3q4ADneLtalMZOZq8CmVR7TIRdeA='} 2024-04-22T06:23:49.189 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi070.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668133', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-22 06:18:48.196392', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEcso0amEbD7LWkysbF+kOSM3IVibZdDd54bL/6XuJSZ7MZk97LGx/Xh4KeuLIRyxLqN83Xjbno3yb2y3nWBpwk='} 2024-04-22T06:23:49.189 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-22T06:23:49.195 INFO:teuthology.task.internal:roles: ubuntu@smithi044.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-04-22T06:23:49.195 INFO:teuthology.task.internal:roles: ubuntu@smithi070.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2024-04-22T06:23:49.195 INFO:teuthology.run_tasks:Running task console_log... 2024-04-22T06:23:49.275 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9dae73a0d0>, signals=[15]) 2024-04-22T06:23:49.275 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-22T06:23:49.282 INFO:teuthology.task.internal:Opening connections... 2024-04-22T06:23:49.282 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi044.front.sepia.ceph.com 2024-04-22T06:23:49.284 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T06:23:49.368 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi070.front.sepia.ceph.com 2024-04-22T06:23:49.369 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T06:23:49.453 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-22T06:23:49.459 DEBUG:teuthology.orchestra.run.smithi044:> uname -m 2024-04-22T06:23:49.478 INFO:teuthology.orchestra.run.smithi044.stdout:x86_64 2024-04-22T06:23:49.478 DEBUG:teuthology.orchestra.run.smithi044:> cat /etc/os-release 2024-04-22T06:23:49.536 INFO:teuthology.orchestra.run.smithi044.stdout:NAME="CentOS Stream" 2024-04-22T06:23:49.536 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION="9" 2024-04-22T06:23:49.536 INFO:teuthology.orchestra.run.smithi044.stdout:ID="centos" 2024-04-22T06:23:49.536 INFO:teuthology.orchestra.run.smithi044.stdout:ID_LIKE="rhel fedora" 2024-04-22T06:23:49.537 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION_ID="9" 2024-04-22T06:23:49.537 INFO:teuthology.orchestra.run.smithi044.stdout:PLATFORM_ID="platform:el9" 2024-04-22T06:23:49.537 INFO:teuthology.orchestra.run.smithi044.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-22T06:23:49.537 INFO:teuthology.orchestra.run.smithi044.stdout:ANSI_COLOR="0;31" 2024-04-22T06:23:49.537 INFO:teuthology.orchestra.run.smithi044.stdout:LOGO="fedora-logo-icon" 2024-04-22T06:23:49.537 INFO:teuthology.orchestra.run.smithi044.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-22T06:23:49.537 INFO:teuthology.orchestra.run.smithi044.stdout:HOME_URL="https://centos.org/" 2024-04-22T06:23:49.537 INFO:teuthology.orchestra.run.smithi044.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-22T06:23:49.537 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-22T06:23:49.537 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-22T06:23:49.538 INFO:teuthology.lock.ops:Updating smithi044.front.sepia.ceph.com on lock server 2024-04-22T06:23:49.574 DEBUG:teuthology.orchestra.run.smithi070:> uname -m 2024-04-22T06:23:49.592 INFO:teuthology.orchestra.run.smithi070.stdout:x86_64 2024-04-22T06:23:49.593 DEBUG:teuthology.orchestra.run.smithi070:> cat /etc/os-release 2024-04-22T06:23:49.650 INFO:teuthology.orchestra.run.smithi070.stdout:NAME="CentOS Stream" 2024-04-22T06:23:49.650 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION="9" 2024-04-22T06:23:49.650 INFO:teuthology.orchestra.run.smithi070.stdout:ID="centos" 2024-04-22T06:23:49.650 INFO:teuthology.orchestra.run.smithi070.stdout:ID_LIKE="rhel fedora" 2024-04-22T06:23:49.650 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION_ID="9" 2024-04-22T06:23:49.650 INFO:teuthology.orchestra.run.smithi070.stdout:PLATFORM_ID="platform:el9" 2024-04-22T06:23:49.650 INFO:teuthology.orchestra.run.smithi070.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-22T06:23:49.650 INFO:teuthology.orchestra.run.smithi070.stdout:ANSI_COLOR="0;31" 2024-04-22T06:23:49.651 INFO:teuthology.orchestra.run.smithi070.stdout:LOGO="fedora-logo-icon" 2024-04-22T06:23:49.651 INFO:teuthology.orchestra.run.smithi070.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-22T06:23:49.651 INFO:teuthology.orchestra.run.smithi070.stdout:HOME_URL="https://centos.org/" 2024-04-22T06:23:49.651 INFO:teuthology.orchestra.run.smithi070.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-22T06:23:49.651 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-22T06:23:49.651 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-22T06:23:49.651 INFO:teuthology.lock.ops:Updating smithi070.front.sepia.ceph.com on lock server 2024-04-22T06:23:49.691 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-22T06:23:49.701 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-22T06:23:49.728 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-22T06:23:49.728 DEBUG:teuthology.orchestra.run.smithi044:> test '!' -e /home/ubuntu/cephtest 2024-04-22T06:23:49.731 DEBUG:teuthology.orchestra.run.smithi070:> test '!' -e /home/ubuntu/cephtest 2024-04-22T06:23:49.747 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-22T06:23:49.753 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-22T06:23:49.753 DEBUG:teuthology.orchestra.run.smithi044:> test -z $(ls -A /var/lib/ceph) 2024-04-22T06:23:49.789 DEBUG:teuthology.orchestra.run.smithi070:> test -z $(ls -A /var/lib/ceph) 2024-04-22T06:23:49.816 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-22T06:23:49.877 INFO:teuthology.run_tasks:Running task kernel... 2024-04-22T06:23:49.891 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-04-22T06:23:49.891 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-04-22T06:23:49.891 DEBUG:teuthology.orchestra.run.smithi044:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-22T06:23:49.891 DEBUG:teuthology.orchestra.run.smithi070:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-22T06:23:49.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T06:23:49.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T06:23:49.909 DEBUG:teuthology.orchestra.run.smithi044:> uname -r 2024-04-22T06:23:49.909 DEBUG:teuthology.orchestra.run.smithi070:> uname -r 2024-04-22T06:23:49.967 INFO:teuthology.orchestra.run.smithi070.stdout:5.14.0-437.el9.x86_64 2024-04-22T06:23:49.967 INFO:teuthology.task.kernel:Running kernel on smithi070: 5.14.0-437.el9.x86_64 2024-04-22T06:23:49.967 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum install -y kernel 2024-04-22T06:23:49.968 INFO:teuthology.orchestra.run.smithi044.stdout:5.14.0-437.el9.x86_64 2024-04-22T06:23:49.968 INFO:teuthology.task.kernel:Running kernel on smithi044: 5.14.0-437.el9.x86_64 2024-04-22T06:23:49.968 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum install -y kernel 2024-04-22T06:23:52.251 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-22T06:23:54.327 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-04-22T06:23:55.388 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - BaseOS 2.1 MB/s | 8.0 MB 00:03 2024-04-22T06:23:57.305 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-04-22T06:23:58.511 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.2 MB 00:00 2024-04-22T06:24:00.296 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 16 kB 00:00 2024-04-22T06:24:02.406 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - CRB 4.1 MB/s | 6.2 MB 00:01 2024-04-22T06:24:03.638 INFO:teuthology.orchestra.run.smithi044.stdout:Extra Packages for Enterprise Linux 6.5 MB/s | 21 MB 00:03 2024-04-22T06:24:04.056 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 16 kB 00:00 2024-04-22T06:24:08.888 INFO:teuthology.orchestra.run.smithi044.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-04-22T06:24:10.869 INFO:teuthology.orchestra.run.smithi044.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-22T06:24:10.869 INFO:teuthology.orchestra.run.smithi044.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-22T06:24:10.949 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-22T06:24:10.950 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-04-22T06:24:10.951 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-22T06:24:11.068 DEBUG:teuthology.orchestra.run.smithi044:> echo no | sudo yum reinstall kernel || true 2024-04-22T06:24:11.588 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:00:03 ago on Mon 22 Apr 2024 06:24:08 AM UTC. 2024-04-22T06:24:11.712 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-22T06:24:11.724 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T06:24:11.725 INFO:teuthology.orchestra.run.smithi044.stdout: Package Architecture Version Repository Size 2024-04-22T06:24:11.725 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T06:24:11.725 INFO:teuthology.orchestra.run.smithi044.stdout:Reinstalling: 2024-04-22T06:24:11.725 INFO:teuthology.orchestra.run.smithi044.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-22T06:24:11.725 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:24:11.725 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-22T06:24:11.725 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T06:24:11.725 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:24:11.725 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 255 k 2024-04-22T06:24:11.725 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 0 2024-04-22T06:24:11.725 INFO:teuthology.orchestra.run.smithi044.stderr:Operation aborted. 2024-04-22T06:24:11.790 INFO:teuthology.orchestra.run.smithi044.stdout:Is this ok [y/N]: 2024-04-22T06:24:11.790 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum reinstall -y kernel || true 2024-04-22T06:24:12.302 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:00:04 ago on Mon 22 Apr 2024 06:24:08 AM UTC. 2024-04-22T06:24:12.423 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-22T06:24:12.424 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T06:24:12.424 INFO:teuthology.orchestra.run.smithi044.stdout: Package Architecture Version Repository Size 2024-04-22T06:24:12.424 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T06:24:12.424 INFO:teuthology.orchestra.run.smithi044.stdout:Reinstalling: 2024-04-22T06:24:12.424 INFO:teuthology.orchestra.run.smithi044.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-22T06:24:12.424 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:24:12.424 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-22T06:24:12.424 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T06:24:12.424 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:24:12.425 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 255 k 2024-04-22T06:24:12.425 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 0 2024-04-22T06:24:12.425 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-04-22T06:24:13.482 INFO:teuthology.orchestra.run.smithi044.stdout:kernel-5.14.0-437.el9.x86_64.rpm 272 kB/s | 255 kB 00:00 2024-04-22T06:24:13.482 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-04-22T06:24:13.482 INFO:teuthology.orchestra.run.smithi044.stdout:Total 241 kB/s | 255 kB 00:01 2024-04-22T06:24:13.502 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-04-22T06:24:13.672 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-04-22T06:24:13.673 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-04-22T06:24:13.678 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-04-22T06:24:13.678 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-04-22T06:24:13.833 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-04-22T06:24:13.907 INFO:teuthology.orchestra.run.smithi044.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-22T06:24:13.907 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T06:24:14.571 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T06:24:14.572 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-22T06:24:15.294 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T06:24:15.294 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:24:15.294 INFO:teuthology.orchestra.run.smithi044.stdout:Reinstalled: 2024-04-22T06:24:15.294 INFO:teuthology.orchestra.run.smithi044.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-22T06:24:15.294 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:24:15.295 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-22T06:24:15.478 DEBUG:teuthology.orchestra.run.smithi044:> rpm -q kernel | sort -rV | head -n 1 2024-04-22T06:24:15.526 INFO:teuthology.orchestra.run.smithi044.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-22T06:24:15.526 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-22T06:24:15.526 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-22T06:24:15.526 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-22T06:24:15.527 DEBUG:teuthology.orchestra.run.smithi044:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-22T06:24:15.557 INFO:teuthology.orchestra.run.smithi044.stdout:ttyS1 2024-04-22T06:24:15.587 DEBUG:teuthology.parallel:result is None 2024-04-22T06:24:23.107 INFO:teuthology.orchestra.run.smithi070.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 21 MB 00:18 2024-04-22T06:24:28.311 INFO:teuthology.orchestra.run.smithi070.stdout:lab-extras 8.6 kB/s | 1.7 kB 00:00 2024-04-22T06:24:30.167 INFO:teuthology.orchestra.run.smithi070.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-22T06:24:30.167 INFO:teuthology.orchestra.run.smithi070.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-22T06:24:30.257 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-04-22T06:24:30.258 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-04-22T06:24:30.258 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-04-22T06:24:30.374 DEBUG:teuthology.orchestra.run.smithi070:> echo no | sudo yum reinstall kernel || true 2024-04-22T06:24:30.871 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:02 ago on Mon 22 Apr 2024 06:24:28 AM UTC. 2024-04-22T06:24:31.039 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-04-22T06:24:31.071 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-22T06:24:31.071 INFO:teuthology.orchestra.run.smithi070.stdout: Package Architecture Version Repository Size 2024-04-22T06:24:31.072 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-22T06:24:31.072 INFO:teuthology.orchestra.run.smithi070.stdout:Reinstalling: 2024-04-22T06:24:31.072 INFO:teuthology.orchestra.run.smithi070.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-22T06:24:31.072 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:24:31.072 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-04-22T06:24:31.072 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-22T06:24:31.072 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:24:31.072 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 255 k 2024-04-22T06:24:31.072 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 0 2024-04-22T06:24:31.072 INFO:teuthology.orchestra.run.smithi070.stderr:Operation aborted. 2024-04-22T06:24:31.184 INFO:teuthology.orchestra.run.smithi070.stdout:Is this ok [y/N]: 2024-04-22T06:24:31.185 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum reinstall -y kernel || true 2024-04-22T06:24:31.672 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:03 ago on Mon 22 Apr 2024 06:24:28 AM UTC. 2024-04-22T06:24:31.788 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-04-22T06:24:31.789 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-22T06:24:31.789 INFO:teuthology.orchestra.run.smithi070.stdout: Package Architecture Version Repository Size 2024-04-22T06:24:31.789 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-22T06:24:31.789 INFO:teuthology.orchestra.run.smithi070.stdout:Reinstalling: 2024-04-22T06:24:31.790 INFO:teuthology.orchestra.run.smithi070.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-22T06:24:31.790 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:24:31.790 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-04-22T06:24:31.790 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-22T06:24:31.790 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:24:31.790 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 255 k 2024-04-22T06:24:31.791 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 0 2024-04-22T06:24:31.791 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-04-22T06:24:32.013 INFO:teuthology.orchestra.run.smithi070.stdout:kernel-5.14.0-437.el9.x86_64.rpm 3.0 MB/s | 255 kB 00:00 2024-04-22T06:24:32.014 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-04-22T06:24:32.014 INFO:teuthology.orchestra.run.smithi070.stdout:Total 1.1 MB/s | 255 kB 00:00 2024-04-22T06:24:32.030 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-04-22T06:24:32.225 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-04-22T06:24:32.225 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-04-22T06:24:32.230 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-04-22T06:24:32.230 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-04-22T06:24:32.369 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-04-22T06:24:32.456 INFO:teuthology.orchestra.run.smithi070.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-22T06:24:32.456 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T06:24:32.964 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T06:24:32.964 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-22T06:24:33.640 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T06:24:33.640 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:24:33.640 INFO:teuthology.orchestra.run.smithi070.stdout:Reinstalled: 2024-04-22T06:24:33.640 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-22T06:24:33.640 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:24:33.640 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-04-22T06:24:33.807 DEBUG:teuthology.orchestra.run.smithi070:> rpm -q kernel | sort -rV | head -n 1 2024-04-22T06:24:33.852 INFO:teuthology.orchestra.run.smithi070.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-22T06:24:33.853 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-22T06:24:33.853 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-22T06:24:33.853 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-22T06:24:33.853 DEBUG:teuthology.orchestra.run.smithi070:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-22T06:24:33.890 INFO:teuthology.orchestra.run.smithi070.stdout:ttyS1 2024-04-22T06:24:33.912 DEBUG:teuthology.parallel:result is None 2024-04-22T06:24:33.912 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-22T06:24:33.919 INFO:teuthology.task.internal:Creating test directory... 2024-04-22T06:24:33.919 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-22T06:24:33.922 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-22T06:24:33.966 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-22T06:24:33.972 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-22T06:24:33.978 INFO:teuthology.task.internal:Creating archive directory... 2024-04-22T06:24:33.979 DEBUG:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-22T06:24:33.981 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-22T06:24:34.038 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-22T06:24:34.044 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-22T06:24:34.044 DEBUG:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-22T06:24:34.047 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-22T06:24:34.078 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T06:24:34.089 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T06:24:34.099 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T06:24:34.108 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T06:24:34.109 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-22T06:24:34.116 INFO:teuthology.task.internal:Configuring sudo... 2024-04-22T06:24:34.116 DEBUG:teuthology.orchestra.run.smithi044:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-22T06:24:34.133 DEBUG:teuthology.orchestra.run.smithi070:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-22T06:24:34.173 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-22T06:24:34.182 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-22T06:24:34.182 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-22T06:24:34.201 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-22T06:24:34.228 DEBUG:teuthology.orchestra.run.smithi044:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T06:24:34.300 DEBUG:teuthology.orchestra.run.smithi044:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T06:24:34.345 DEBUG:teuthology.orchestra.run.smithi044:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T06:24:34.431 DEBUG:teuthology.orchestra.run.smithi044:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T06:24:34.472 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T06:24:34.472 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-22T06:24:34.539 DEBUG:teuthology.orchestra.run.smithi070:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T06:24:34.579 DEBUG:teuthology.orchestra.run.smithi070:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T06:24:34.623 DEBUG:teuthology.orchestra.run.smithi070:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T06:24:34.706 DEBUG:teuthology.orchestra.run.smithi070:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T06:24:34.743 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T06:24:34.743 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-22T06:24:34.806 DEBUG:teuthology.orchestra.run.smithi044:> sudo service rsyslog restart 2024-04-22T06:24:34.808 DEBUG:teuthology.orchestra.run.smithi070:> sudo service rsyslog restart 2024-04-22T06:24:34.847 INFO:teuthology.orchestra.run.smithi044.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-22T06:24:34.877 INFO:teuthology.orchestra.run.smithi070.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-22T06:24:35.336 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-22T06:24:35.344 INFO:teuthology.task.internal:Starting timer... 2024-04-22T06:24:35.344 INFO:teuthology.run_tasks:Running task pcp... 2024-04-22T06:24:35.369 INFO:teuthology.run_tasks:Running task selinux... 2024-04-22T06:24:35.377 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-22T06:24:35.377 DEBUG:teuthology.orchestra.run.smithi044:> sudo service auditd rotate 2024-04-22T06:24:35.422 INFO:teuthology.orchestra.run.smithi044.stdout:Rotating logs: 2024-04-22T06:24:35.424 DEBUG:teuthology.orchestra.run.smithi070:> sudo service auditd rotate 2024-04-22T06:24:35.485 INFO:teuthology.orchestra.run.smithi070.stdout:Rotating logs: 2024-04-22T06:24:35.486 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-22T06:24:35.487 DEBUG:teuthology.orchestra.run.smithi044:> /usr/sbin/getenforce 2024-04-22T06:24:35.511 INFO:teuthology.orchestra.run.smithi044.stdout:Permissive 2024-04-22T06:24:35.511 DEBUG:teuthology.orchestra.run.smithi070:> /usr/sbin/getenforce 2024-04-22T06:24:35.533 INFO:teuthology.orchestra.run.smithi070.stdout:Permissive 2024-04-22T06:24:35.533 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi044.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi070.front.sepia.ceph.com': 'permissive'} 2024-04-22T06:24:35.534 DEBUG:teuthology.orchestra.run.smithi044:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-22T06:24:35.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T06:24:35.579 DEBUG:teuthology.orchestra.run.smithi070:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-22T06:24:35.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T06:24:35.602 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-22T06:24:35.603 DEBUG:teuthology.orchestra.run.smithi044:> sudo /usr/sbin/setenforce permissive 2024-04-22T06:24:35.645 DEBUG:teuthology.orchestra.run.smithi070:> sudo /usr/sbin/setenforce permissive 2024-04-22T06:24:35.668 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-22T06:24:35.678 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-22T06:24:35.694 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-22T06:24:35.695 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi044.front.sepia.ceph.com,smithi070.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-22T06:28:06.160 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi044.front.sepia.ceph.com'), Remote(name='ubuntu@smithi070.front.sepia.ceph.com')] 2024-04-22T06:28:06.161 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-04-22T06:28:06.163 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T06:28:06.275 DEBUG:teuthology.orchestra.run.smithi044:> true 2024-04-22T06:28:06.353 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-04-22T06:28:06.353 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-04-22T06:28:06.354 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T06:28:06.439 DEBUG:teuthology.orchestra.run.smithi070:> true 2024-04-22T06:28:06.517 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-04-22T06:28:06.517 INFO:teuthology.run_tasks:Running task clock... 2024-04-22T06:28:06.529 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-22T06:28:06.529 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-22T06:28:06.530 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T06:28:06.532 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-22T06:28:06.533 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T06:28:06.559 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-22T06:28:06.573 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-22T06:28:06.584 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-22T06:28:06.597 INFO:teuthology.orchestra.run.smithi044.stderr:sudo: ntpd: command not found 2024-04-22T06:28:06.597 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-22T06:28:06.607 INFO:teuthology.orchestra.run.smithi044.stdout:506 Cannot talk to daemon 2024-04-22T06:28:06.620 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: ntpd: command not found 2024-04-22T06:28:06.622 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-22T06:28:06.630 INFO:teuthology.orchestra.run.smithi070.stdout:506 Cannot talk to daemon 2024-04-22T06:28:06.636 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-22T06:28:06.643 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-22T06:28:06.656 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-22T06:28:06.697 INFO:teuthology.orchestra.run.smithi044.stderr:bash: line 1: ntpq: command not found 2024-04-22T06:28:06.701 INFO:teuthology.orchestra.run.smithi044.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-22T06:28:06.701 INFO:teuthology.orchestra.run.smithi044.stdout:=============================================================================== 2024-04-22T06:28:06.701 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T06:28:06.702 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T06:28:06.702 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T06:28:06.702 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T06:28:06.711 INFO:teuthology.orchestra.run.smithi070.stderr:bash: line 1: ntpq: command not found 2024-04-22T06:28:06.713 INFO:teuthology.orchestra.run.smithi070.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-22T06:28:06.713 INFO:teuthology.orchestra.run.smithi070.stdout:=============================================================================== 2024-04-22T06:28:06.713 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T06:28:06.714 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T06:28:06.714 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T06:28:06.714 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T06:28:06.715 INFO:teuthology.run_tasks:Running task install... 2024-04-22T06:28:06.727 DEBUG:teuthology.task.install:project ceph 2024-04-22T06:28:06.727 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '86f7587a5a09af35f0895e1a2d08527638fae697'}} 2024-04-22T06:28:06.727 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '86f7587a5a09af35f0895e1a2d08527638fae697'} 2024-04-22T06:28:06.728 INFO:teuthology.task.install:Using flavor: default 2024-04-22T06:28:06.733 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-22T06:28:06.733 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-22T06:28:06.734 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '86f7587a5a09af35f0895e1a2d08527638fae697', 'tag': None, 'wait_for_package': False} 2024-04-22T06:28:06.735 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=86f7587a5a09af35f0895e1a2d08527638fae697 2024-04-22T06:28:06.737 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '86f7587a5a09af35f0895e1a2d08527638fae697', 'tag': None, 'wait_for_package': False} 2024-04-22T06:28:06.737 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=86f7587a5a09af35f0895e1a2d08527638fae697 2024-04-22T06:28:06.975 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/flavors/default/ 2024-04-22T06:28:06.975 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.g86f7587a 2024-04-22T06:28:06.992 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/flavors/default/ 2024-04-22T06:28:06.992 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.g86f7587a 2024-04-22T06:28:07.167 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-22T06:28:07.167 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T06:28:07.167 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-22T06:28:07.182 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-22T06:28:07.182 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T06:28:07.182 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-22T06:28:07.194 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-22T06:28:07.194 DEBUG:teuthology.orchestra.run.smithi044:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/86f7587a5a09af35f0895e1a2d08527638fae697/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-22T06:28:07.209 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-22T06:28:07.209 DEBUG:teuthology.orchestra.run.smithi070:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/86f7587a5a09af35f0895e1a2d08527638fae697/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-22T06:28:07.260 DEBUG:teuthology.orchestra.run.smithi044:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-22T06:28:07.274 DEBUG:teuthology.orchestra.run.smithi070:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-22T06:28:07.335 DEBUG:teuthology.orchestra.run.smithi044:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-22T06:28:07.349 DEBUG:teuthology.orchestra.run.smithi070:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-22T06:28:07.399 INFO:teuthology.orchestra.run.smithi044.stdout:check_obsoletes = 1 2024-04-22T06:28:07.401 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum clean all 2024-04-22T06:28:07.412 INFO:teuthology.orchestra.run.smithi070.stdout:check_obsoletes = 1 2024-04-22T06:28:07.414 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum clean all 2024-04-22T06:28:07.634 INFO:teuthology.orchestra.run.smithi044.stdout:48 files removed 2024-04-22T06:28:07.637 INFO:teuthology.orchestra.run.smithi070.stdout:48 files removed 2024-04-22T06:28:07.661 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-22T06:28:07.664 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-22T06:28:08.401 INFO:teuthology.orchestra.run.smithi070.stdout:ceph packages for x86_64 175 kB/s | 83 kB 00:00 2024-04-22T06:28:08.401 INFO:teuthology.orchestra.run.smithi044.stdout:ceph packages for x86_64 179 kB/s | 83 kB 00:00 2024-04-22T06:28:08.770 INFO:teuthology.orchestra.run.smithi044.stdout:ceph noarch packages 36 kB/s | 12 kB 00:00 2024-04-22T06:28:08.770 INFO:teuthology.orchestra.run.smithi070.stdout:ceph noarch packages 36 kB/s | 12 kB 00:00 2024-04-22T06:28:09.136 INFO:teuthology.orchestra.run.smithi070.stdout:ceph source packages 5.6 kB/s | 1.9 kB 00:00 2024-04-22T06:28:09.203 INFO:teuthology.orchestra.run.smithi044.stdout:ceph source packages 4.7 kB/s | 1.9 kB 00:00 2024-04-22T06:28:10.061 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - BaseOS 9.8 MB/s | 8.0 MB 00:00 2024-04-22T06:28:10.154 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - BaseOS 8.2 MB/s | 8.0 MB 00:00 2024-04-22T06:28:11.903 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - AppStream 27 MB/s | 19 MB 00:00 2024-04-22T06:28:12.012 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-04-22T06:28:16.212 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - CRB 8.0 MB/s | 6.2 MB 00:00 2024-04-22T06:28:16.380 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - CRB 8.8 MB/s | 6.2 MB 00:00 2024-04-22T06:28:17.730 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 16 kB 00:00 2024-04-22T06:28:18.048 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-04-22T06:28:19.815 INFO:teuthology.orchestra.run.smithi044.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-22T06:28:24.983 INFO:teuthology.orchestra.run.smithi044.stdout:lab-extras 43 kB/s | 1.7 kB 00:00 2024-04-22T06:28:27.177 INFO:teuthology.orchestra.run.smithi044.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-22T06:28:27.177 INFO:teuthology.orchestra.run.smithi044.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-22T06:28:27.189 INFO:teuthology.orchestra.run.smithi044.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-22T06:28:27.190 INFO:teuthology.orchestra.run.smithi044.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-22T06:28:27.190 INFO:teuthology.orchestra.run.smithi044.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-22T06:28:27.191 INFO:teuthology.orchestra.run.smithi044.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-22T06:28:27.193 INFO:teuthology.orchestra.run.smithi044.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-22T06:28:27.195 INFO:teuthology.orchestra.run.smithi044.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-22T06:28:27.196 INFO:teuthology.orchestra.run.smithi044.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-22T06:28:27.261 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-22T06:28:27.270 INFO:teuthology.orchestra.run.smithi044.stdout:======================================================================================= 2024-04-22T06:28:27.270 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-04-22T06:28:27.270 INFO:teuthology.orchestra.run.smithi044.stdout:======================================================================================= 2024-04-22T06:28:27.270 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-04-22T06:28:27.270 INFO:teuthology.orchestra.run.smithi044.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-22T06:28:27.270 INFO:teuthology.orchestra.run.smithi044.stdout: ceph x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 6.6 k 2024-04-22T06:28:27.270 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-base x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 5.4 M 2024-04-22T06:28:27.271 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-fuse x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 890 k 2024-04-22T06:28:27.271 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 145 k 2024-04-22T06:28:27.271 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 1.5 M 2024-04-22T06:28:27.271 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 142 k 2024-04-22T06:28:27.271 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 3.5 M 2024-04-22T06:28:27.271 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 7.4 M 2024-04-22T06:28:27.271 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 49 k 2024-04-22T06:28:27.271 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-radosgw x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 11 M 2024-04-22T06:28:27.271 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-test x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 49 M 2024-04-22T06:28:27.271 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-volume noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 273 k 2024-04-22T06:28:27.271 INFO:teuthology.orchestra.run.smithi044.stdout: cephadm noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 763 k 2024-04-22T06:28:27.271 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-mirror x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 225 k 2024-04-22T06:28:27.271 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-shell x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 27 k 2024-04-22T06:28:27.272 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-top noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 18 k 2024-04-22T06:28:27.272 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-22T06:28:27.272 INFO:teuthology.orchestra.run.smithi044.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-22T06:28:27.272 INFO:teuthology.orchestra.run.smithi044.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-22T06:28:27.272 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs-devel x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 32 k 2024-04-22T06:28:27.272 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs2 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 747 k 2024-04-22T06:28:27.272 INFO:teuthology.orchestra.run.smithi044.stdout: librados-devel x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 127 k 2024-04-22T06:28:27.272 INFO:teuthology.orchestra.run.smithi044.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-22T06:28:27.272 INFO:teuthology.orchestra.run.smithi044.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-22T06:28:27.272 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-22T06:28:27.272 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-22T06:28:27.272 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-22T06:28:27.272 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-22T06:28:27.273 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cephfs x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 162 k 2024-04-22T06:28:27.273 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rados x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 320 k 2024-04-22T06:28:27.273 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rbd x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 298 k 2024-04-22T06:28:27.273 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rgw x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 100 k 2024-04-22T06:28:27.273 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-fuse x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 85 k 2024-04-22T06:28:27.273 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-mirror x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 3.1 M 2024-04-22T06:28:27.273 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-nbd x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 171 k 2024-04-22T06:28:27.273 INFO:teuthology.orchestra.run.smithi044.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-22T06:28:27.273 INFO:teuthology.orchestra.run.smithi044.stdout:Upgrading: 2024-04-22T06:28:27.273 INFO:teuthology.orchestra.run.smithi044.stdout: librados2 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 3.4 M 2024-04-22T06:28:27.273 INFO:teuthology.orchestra.run.smithi044.stdout: librbd1 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 3.1 M 2024-04-22T06:28:27.273 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-04-22T06:28:27.273 INFO:teuthology.orchestra.run.smithi044.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-22T06:28:27.274 INFO:teuthology.orchestra.run.smithi044.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-22T06:28:27.274 INFO:teuthology.orchestra.run.smithi044.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-22T06:28:27.274 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-common x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 21 M 2024-04-22T06:28:27.274 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 34 k 2024-04-22T06:28:27.274 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mds x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 2.4 M 2024-04-22T06:28:27.274 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 249 k 2024-04-22T06:28:27.274 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mon x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 4.7 M 2024-04-22T06:28:27.274 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-osd x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 17 M 2024-04-22T06:28:27.274 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 16 k 2024-04-22T06:28:27.274 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-selinux x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 25 k 2024-04-22T06:28:27.274 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-22T06:28:27.274 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-22T06:28:27.274 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-22T06:28:27.275 INFO:teuthology.orchestra.run.smithi044.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-22T06:28:27.275 INFO:teuthology.orchestra.run.smithi044.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-22T06:28:27.275 INFO:teuthology.orchestra.run.smithi044.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-22T06:28:27.275 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-22T06:28:27.275 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-22T06:28:27.275 INFO:teuthology.orchestra.run.smithi044.stdout: libcephsqlite x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 164 k 2024-04-22T06:28:27.275 INFO:teuthology.orchestra.run.smithi044.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-22T06:28:27.275 INFO:teuthology.orchestra.run.smithi044.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-22T06:28:27.275 INFO:teuthology.orchestra.run.smithi044.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-22T06:28:27.275 INFO:teuthology.orchestra.run.smithi044.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-22T06:28:27.275 INFO:teuthology.orchestra.run.smithi044.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-22T06:28:27.275 INFO:teuthology.orchestra.run.smithi044.stdout: libradosstriper1 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 500 k 2024-04-22T06:28:27.275 INFO:teuthology.orchestra.run.smithi044.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-22T06:28:27.276 INFO:teuthology.orchestra.run.smithi044.stdout: librgw2 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 5.4 M 2024-04-22T06:28:27.276 INFO:teuthology.orchestra.run.smithi044.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-22T06:28:27.276 INFO:teuthology.orchestra.run.smithi044.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-22T06:28:27.276 INFO:teuthology.orchestra.run.smithi044.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-22T06:28:27.276 INFO:teuthology.orchestra.run.smithi044.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-22T06:28:27.276 INFO:teuthology.orchestra.run.smithi044.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-22T06:28:27.276 INFO:teuthology.orchestra.run.smithi044.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-22T06:28:27.276 INFO:teuthology.orchestra.run.smithi044.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-22T06:28:27.276 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-22T06:28:27.276 INFO:teuthology.orchestra.run.smithi044.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-22T06:28:27.276 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-22T06:28:27.276 INFO:teuthology.orchestra.run.smithi044.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-22T06:28:27.276 INFO:teuthology.orchestra.run.smithi044.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-22T06:28:27.276 INFO:teuthology.orchestra.run.smithi044.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-22T06:28:27.277 INFO:teuthology.orchestra.run.smithi044.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-22T06:28:27.277 INFO:teuthology.orchestra.run.smithi044.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-22T06:28:27.277 INFO:teuthology.orchestra.run.smithi044.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-22T06:28:27.277 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-22T06:28:27.277 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 45 k 2024-04-22T06:28:27.277 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-common x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 135 k 2024-04-22T06:28:27.277 INFO:teuthology.orchestra.run.smithi044.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-22T06:28:27.277 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-22T06:28:27.277 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-22T06:28:27.277 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-22T06:28:27.277 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-22T06:28:27.277 INFO:teuthology.orchestra.run.smithi044.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-22T06:28:27.277 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-22T06:28:27.278 INFO:teuthology.orchestra.run.smithi044.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-22T06:28:27.278 INFO:teuthology.orchestra.run.smithi044.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-22T06:28:27.278 INFO:teuthology.orchestra.run.smithi044.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-22T06:28:27.278 INFO:teuthology.orchestra.run.smithi044.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-22T06:28:27.278 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-22T06:28:27.278 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-22T06:28:27.278 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-22T06:28:27.278 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-22T06:28:27.278 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-22T06:28:27.278 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-22T06:28:27.278 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-22T06:28:27.278 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-22T06:28:27.278 INFO:teuthology.orchestra.run.smithi044.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-22T06:28:27.279 INFO:teuthology.orchestra.run.smithi044.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-22T06:28:27.279 INFO:teuthology.orchestra.run.smithi044.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-22T06:28:27.279 INFO:teuthology.orchestra.run.smithi044.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-22T06:28:27.279 INFO:teuthology.orchestra.run.smithi044.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-22T06:28:27.279 INFO:teuthology.orchestra.run.smithi044.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-22T06:28:27.279 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-22T06:28:27.279 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-22T06:28:27.280 INFO:teuthology.orchestra.run.smithi044.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-22T06:28:27.280 INFO:teuthology.orchestra.run.smithi044.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-22T06:28:27.280 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-22T06:28:27.280 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-22T06:28:27.280 INFO:teuthology.orchestra.run.smithi044.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-22T06:28:27.280 INFO:teuthology.orchestra.run.smithi044.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-22T06:28:27.280 INFO:teuthology.orchestra.run.smithi044.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-22T06:28:27.280 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-22T06:28:27.280 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-22T06:28:27.280 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-22T06:28:27.280 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-22T06:28:27.280 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-22T06:28:27.280 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-22T06:28:27.280 INFO:teuthology.orchestra.run.smithi044.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-22T06:28:27.281 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-22T06:28:27.281 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-22T06:28:27.281 INFO:teuthology.orchestra.run.smithi044.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-22T06:28:27.281 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-22T06:28:27.281 INFO:teuthology.orchestra.run.smithi044.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-22T06:28:27.281 INFO:teuthology.orchestra.run.smithi044.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-22T06:28:27.281 INFO:teuthology.orchestra.run.smithi044.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-22T06:28:27.281 INFO:teuthology.orchestra.run.smithi044.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-22T06:28:27.281 INFO:teuthology.orchestra.run.smithi044.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-22T06:28:27.281 INFO:teuthology.orchestra.run.smithi044.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-22T06:28:27.281 INFO:teuthology.orchestra.run.smithi044.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-22T06:28:27.281 INFO:teuthology.orchestra.run.smithi044.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-22T06:28:27.281 INFO:teuthology.orchestra.run.smithi044.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-22T06:28:27.282 INFO:teuthology.orchestra.run.smithi044.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-22T06:28:27.282 INFO:teuthology.orchestra.run.smithi044.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-22T06:28:27.282 INFO:teuthology.orchestra.run.smithi044.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-22T06:28:27.282 INFO:teuthology.orchestra.run.smithi044.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-22T06:28:27.282 INFO:teuthology.orchestra.run.smithi044.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-22T06:28:27.282 INFO:teuthology.orchestra.run.smithi044.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-22T06:28:27.282 INFO:teuthology.orchestra.run.smithi044.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-22T06:28:27.282 INFO:teuthology.orchestra.run.smithi044.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-22T06:28:27.282 INFO:teuthology.orchestra.run.smithi044.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-22T06:28:27.282 INFO:teuthology.orchestra.run.smithi044.stdout:Installing weak dependencies: 2024-04-22T06:28:27.282 INFO:teuthology.orchestra.run.smithi044.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-22T06:28:27.282 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:28:27.282 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-22T06:28:27.282 INFO:teuthology.orchestra.run.smithi044.stdout:======================================================================================= 2024-04-22T06:28:27.283 INFO:teuthology.orchestra.run.smithi044.stdout:Install 148 Packages 2024-04-22T06:28:27.283 INFO:teuthology.orchestra.run.smithi044.stdout:Upgrade 2 Packages 2024-04-22T06:28:27.283 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:28:27.283 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 218 M 2024-04-22T06:28:27.283 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-04-22T06:28:27.892 INFO:teuthology.orchestra.run.smithi044.stdout:(1/150): ceph-19.0.0-3156.g86f7587a.el9.x86_64. 33 kB/s | 6.6 kB 00:00 2024-04-22T06:28:28.251 INFO:teuthology.orchestra.run.smithi044.stdout:(2/150): ceph-fuse-19.0.0-3156.g86f7587a.el9.x8 2.4 MB/s | 890 kB 00:00 2024-04-22T06:28:28.326 INFO:teuthology.orchestra.run.smithi044.stdout:(3/150): ceph-base-19.0.0-3156.g86f7587a.el9.x8 8.5 MB/s | 5.4 MB 00:00 2024-04-22T06:28:28.351 INFO:teuthology.orchestra.run.smithi044.stdout:(4/150): ceph-immutable-object-cache-19.0.0-315 1.4 MB/s | 145 kB 00:00 2024-04-22T06:28:28.543 INFO:teuthology.orchestra.run.smithi044.stdout:(5/150): ceph-mds-19.0.0-3156.g86f7587a.el9.x86 11 MB/s | 2.4 MB 00:00 2024-04-22T06:28:28.669 INFO:teuthology.orchestra.run.smithi044.stdout:(6/150): ceph-mgr-19.0.0-3156.g86f7587a.el9.x86 4.7 MB/s | 1.5 MB 00:00 2024-04-22T06:28:29.444 INFO:teuthology.orchestra.run.smithi044.stdout:(7/150): ceph-common-19.0.0-3156.g86f7587a.el9. 12 MB/s | 21 MB 00:01 2024-04-22T06:28:29.811 INFO:teuthology.orchestra.run.smithi044.stdout:(8/150): ceph-mon-19.0.0-3156.g86f7587a.el9.x86 3.7 MB/s | 4.7 MB 00:01 2024-04-22T06:28:29.902 INFO:teuthology.orchestra.run.smithi044.stdout:(9/150): ceph-selinux-19.0.0-3156.g86f7587a.el9 275 kB/s | 25 kB 00:00 2024-04-22T06:28:30.262 INFO:teuthology.orchestra.run.smithi070.stdout:Extra Packages for Enterprise Linux 1.7 MB/s | 21 MB 00:12 2024-04-22T06:28:30.403 INFO:teuthology.orchestra.run.smithi044.stdout:(10/150): ceph-radosgw-19.0.0-3156.g86f7587a.el 11 MB/s | 11 MB 00:00 2024-04-22T06:28:30.453 INFO:teuthology.orchestra.run.smithi044.stdout:(11/150): cephfs-mirror-19.0.0-3156.g86f7587a.e 4.4 MB/s | 225 kB 00:00 2024-04-22T06:28:30.620 INFO:teuthology.orchestra.run.smithi044.stdout:(12/150): ceph-osd-19.0.0-3156.g86f7587a.el9.x8 8.6 MB/s | 17 MB 00:01 2024-04-22T06:28:30.645 INFO:teuthology.orchestra.run.smithi044.stdout:(13/150): cephfs-shell-19.0.0-3156.g86f7587a.el 141 kB/s | 27 kB 00:00 2024-04-22T06:28:30.671 INFO:teuthology.orchestra.run.smithi044.stdout:(14/150): libcephfs-devel-19.0.0-3156.g86f7587a 634 kB/s | 32 kB 00:00 2024-04-22T06:28:30.738 INFO:teuthology.orchestra.run.smithi044.stdout:(15/150): libcephsqlite-19.0.0-3156.g86f7587a.e 2.4 MB/s | 164 kB 00:00 2024-04-22T06:28:30.763 INFO:teuthology.orchestra.run.smithi044.stdout:(16/150): libcephfs2-19.0.0-3156.g86f7587a.el9. 6.2 MB/s | 747 kB 00:00 2024-04-22T06:28:30.796 INFO:teuthology.orchestra.run.smithi044.stdout:(17/150): librados-devel-19.0.0-3156.g86f7587a. 2.1 MB/s | 127 kB 00:00 2024-04-22T06:28:30.830 INFO:teuthology.orchestra.run.smithi044.stdout:(18/150): libradosstriper1-19.0.0-3156.g86f7587 7.3 MB/s | 500 kB 00:00 2024-04-22T06:28:30.914 INFO:teuthology.orchestra.run.smithi044.stdout:(19/150): python3-ceph-argparse-19.0.0-3156.g86 541 kB/s | 45 kB 00:00 2024-04-22T06:28:31.006 INFO:teuthology.orchestra.run.smithi044.stdout:(20/150): python3-ceph-common-19.0.0-3156.g86f7 1.4 MB/s | 135 kB 00:00 2024-04-22T06:28:31.114 INFO:teuthology.orchestra.run.smithi044.stdout:(21/150): python3-cephfs-19.0.0-3156.g86f7587a. 1.5 MB/s | 162 kB 00:00 2024-04-22T06:28:31.189 INFO:teuthology.orchestra.run.smithi044.stdout:(22/150): librgw2-19.0.0-3156.g86f7587a.el9.x86 14 MB/s | 5.4 MB 00:00 2024-04-22T06:28:31.215 INFO:teuthology.orchestra.run.smithi044.stdout:(23/150): python3-rados-19.0.0-3156.g86f7587a.e 3.1 MB/s | 320 kB 00:00 2024-04-22T06:28:31.248 INFO:teuthology.orchestra.run.smithi044.stdout:(24/150): python3-rbd-19.0.0-3156.g86f7587a.el9 5.0 MB/s | 298 kB 00:00 2024-04-22T06:28:31.274 INFO:teuthology.orchestra.run.smithi044.stdout:(25/150): python3-rgw-19.0.0-3156.g86f7587a.el9 1.7 MB/s | 100 kB 00:00 2024-04-22T06:28:31.307 INFO:teuthology.orchestra.run.smithi044.stdout:(26/150): rbd-fuse-19.0.0-3156.g86f7587a.el9.x8 1.4 MB/s | 85 kB 00:00 2024-04-22T06:28:31.391 INFO:teuthology.orchestra.run.smithi044.stdout:(27/150): rbd-nbd-19.0.0-3156.g86f7587a.el9.x86 2.0 MB/s | 171 kB 00:00 2024-04-22T06:28:31.466 INFO:teuthology.orchestra.run.smithi044.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3156.g 458 kB/s | 34 kB 00:00 2024-04-22T06:28:31.533 INFO:teuthology.orchestra.run.smithi044.stdout:(29/150): rbd-mirror-19.0.0-3156.g86f7587a.el9. 12 MB/s | 3.1 MB 00:00 2024-04-22T06:28:31.559 INFO:teuthology.orchestra.run.smithi044.stdout:(30/150): ceph-mgr-cephadm-19.0.0-3156.g86f7587 1.5 MB/s | 142 kB 00:00 2024-04-22T06:28:32.017 INFO:teuthology.orchestra.run.smithi044.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3156.g86f75 7.3 MB/s | 3.5 MB 00:00 2024-04-22T06:28:32.092 INFO:teuthology.orchestra.run.smithi044.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3156.g86 3.3 MB/s | 249 kB 00:00 2024-04-22T06:28:32.226 INFO:teuthology.orchestra.run.smithi044.stdout:(33/150): ceph-mgr-diskprediction-local-19.0.0- 11 MB/s | 7.4 MB 00:00 2024-04-22T06:28:32.251 INFO:teuthology.orchestra.run.smithi044.stdout:(34/150): ceph-mgr-rook-19.0.0-3156.g86f7587a.e 311 kB/s | 49 kB 00:00 2024-04-22T06:28:32.285 INFO:teuthology.orchestra.run.smithi044.stdout:(35/150): ceph-prometheus-alerts-19.0.0-3156.g8 283 kB/s | 16 kB 00:00 2024-04-22T06:28:32.310 INFO:teuthology.orchestra.run.smithi044.stdout:(36/150): ceph-volume-19.0.0-3156.g86f7587a.el9 4.6 MB/s | 273 kB 00:00 2024-04-22T06:28:32.369 INFO:teuthology.orchestra.run.smithi044.stdout:(37/150): cephadm-19.0.0-3156.g86f7587a.el9.noa 8.9 MB/s | 763 kB 00:00 2024-04-22T06:28:32.394 INFO:teuthology.orchestra.run.smithi044.stdout:(38/150): cephfs-top-19.0.0-3156.g86f7587a.el9. 218 kB/s | 18 kB 00:00 2024-04-22T06:28:32.778 INFO:teuthology.orchestra.run.smithi044.stdout:(39/150): fuse-2.9.9-15.el9.x86_64.rpm 196 kB/s | 80 kB 00:00 2024-04-22T06:28:33.045 INFO:teuthology.orchestra.run.smithi044.stdout:(40/150): libquadmath-11.4.1-3.el9.x86_64.rpm 733 kB/s | 195 kB 00:00 2024-04-22T06:28:33.187 INFO:teuthology.orchestra.run.smithi044.stdout:(41/150): libgfortran-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 803 kB 00:00 2024-04-22T06:28:33.754 INFO:teuthology.orchestra.run.smithi044.stdout:(42/150): ceph-test-19.0.0-3156.g86f7587a.el9.x 13 MB/s | 49 MB 00:03 2024-04-22T06:28:33.779 INFO:teuthology.orchestra.run.smithi044.stdout:(43/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 345 kB/s | 253 kB 00:00 2024-04-22T06:28:33.946 INFO:teuthology.orchestra.run.smithi044.stdout:(44/150): python3-cryptography-36.0.1-4.el9.x86 1.6 MB/s | 1.2 MB 00:00 2024-04-22T06:28:33.972 INFO:teuthology.orchestra.run.smithi044.stdout:(45/150): python3-pycparser-2.20-6.el9.noarch.r 704 kB/s | 135 kB 00:00 2024-04-22T06:28:34.047 INFO:teuthology.orchestra.run.smithi044.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-04-22T06:28:34.080 INFO:teuthology.orchestra.run.smithi044.stdout:(47/150): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-04-22T06:28:34.289 INFO:teuthology.orchestra.run.smithi044.stdout:(48/150): python3-ply-3.11-14.el9.noarch.rpm 199 kB/s | 106 kB 00:00 2024-04-22T06:28:34.322 INFO:teuthology.orchestra.run.smithi044.stdout:(49/150): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-04-22T06:28:34.390 INFO:teuthology.orchestra.run.smithi044.stdout:(50/150): bison-3.7.4-5.el9.x86_64.rpm 3.0 MB/s | 943 kB 00:00 2024-04-22T06:28:34.432 INFO:teuthology.orchestra.run.smithi044.stdout:(51/150): elfutils-libelf-devel-0.190-2.el9.x86 198 kB/s | 21 kB 00:00 2024-04-22T06:28:34.473 INFO:teuthology.orchestra.run.smithi044.stdout:(52/150): boost-program-options-1.75.0-8.el9.x8 580 kB/s | 107 kB 00:00 2024-04-22T06:28:34.499 INFO:teuthology.orchestra.run.smithi044.stdout:(53/150): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-04-22T06:28:34.526 INFO:teuthology.orchestra.run.smithi044.stdout:(54/150): flexiblas-3.0.4-8.el9.x86_64.rpm 354 kB/s | 33 kB 00:00 2024-04-22T06:28:34.599 INFO:teuthology.orchestra.run.smithi044.stdout:(55/150): flexiblas-openblas-openmp-3.0.4-8.el9 181 kB/s | 18 kB 00:00 2024-04-22T06:28:34.624 INFO:teuthology.orchestra.run.smithi044.stdout:(56/150): libaio-devel-0.3.111-13.el9.x86_64.rp 111 kB/s | 11 kB 00:00 2024-04-22T06:28:34.708 INFO:teuthology.orchestra.run.smithi044.stdout:(57/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-04-22T06:28:34.725 INFO:teuthology.orchestra.run.smithi044.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 451 kB/s | 45 kB 00:00 2024-04-22T06:28:34.842 INFO:teuthology.orchestra.run.smithi044.stdout:(59/150): librdkafka-1.6.1-102.el9.x86_64.rpm 4.8 MB/s | 662 kB 00:00 2024-04-22T06:28:34.876 INFO:teuthology.orchestra.run.smithi044.stdout:(60/150): libtool-2.4.6-45.el9.x86_64.rpm 3.8 MB/s | 582 kB 00:00 2024-04-22T06:28:34.943 INFO:teuthology.orchestra.run.smithi044.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 254 kB/s | 25 kB 00:00 2024-04-22T06:28:34.968 INFO:teuthology.orchestra.run.smithi044.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 517 kB/s | 47 kB 00:00 2024-04-22T06:28:35.052 INFO:teuthology.orchestra.run.smithi044.stdout:(63/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-04-22T06:28:35.077 INFO:teuthology.orchestra.run.smithi044.stdout:(64/150): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-04-22T06:28:35.152 INFO:teuthology.orchestra.run.smithi044.stdout:(65/150): openblas-0.3.21-2.el9.x86_64.rpm 352 kB/s | 35 kB 00:00 2024-04-22T06:28:35.227 INFO:teuthology.orchestra.run.smithi044.stdout:(66/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.0 MB/s | 3.0 MB 00:00 2024-04-22T06:28:35.294 INFO:teuthology.orchestra.run.smithi070.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-04-22T06:28:35.353 INFO:teuthology.orchestra.run.smithi044.stdout:(67/150): postgresql-13.14-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-22T06:28:36.571 INFO:teuthology.orchestra.run.smithi044.stdout:(68/150): postgresql-contrib-13.14-1.el9.x86_64 748 kB/s | 911 kB 00:01 2024-04-22T06:28:36.647 INFO:teuthology.orchestra.run.smithi044.stdout:(69/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 2.8 MB/s | 4.1 MB 00:01 2024-04-22T06:28:36.864 INFO:teuthology.orchestra.run.smithi044.stdout:(70/150): postgresql-private-libs-13.14-1.el9.x 456 kB/s | 133 kB 00:00 2024-04-22T06:28:37.487 INFO:teuthology.orchestra.run.smithi070.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-22T06:28:37.487 INFO:teuthology.orchestra.run.smithi070.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-22T06:28:37.498 INFO:teuthology.orchestra.run.smithi070.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-22T06:28:37.499 INFO:teuthology.orchestra.run.smithi070.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-22T06:28:37.499 INFO:teuthology.orchestra.run.smithi070.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-22T06:28:37.500 INFO:teuthology.orchestra.run.smithi070.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-22T06:28:37.502 INFO:teuthology.orchestra.run.smithi070.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-22T06:28:37.504 INFO:teuthology.orchestra.run.smithi070.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-22T06:28:37.505 INFO:teuthology.orchestra.run.smithi070.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-22T06:28:37.564 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-04-22T06:28:37.573 INFO:teuthology.orchestra.run.smithi070.stdout:======================================================================================= 2024-04-22T06:28:37.573 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-04-22T06:28:37.573 INFO:teuthology.orchestra.run.smithi070.stdout:======================================================================================= 2024-04-22T06:28:37.573 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-04-22T06:28:37.573 INFO:teuthology.orchestra.run.smithi070.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-22T06:28:37.574 INFO:teuthology.orchestra.run.smithi070.stdout: ceph x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 6.6 k 2024-04-22T06:28:37.574 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-base x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 5.4 M 2024-04-22T06:28:37.574 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-fuse x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 890 k 2024-04-22T06:28:37.574 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 145 k 2024-04-22T06:28:37.574 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 1.5 M 2024-04-22T06:28:37.574 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 142 k 2024-04-22T06:28:37.574 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 3.5 M 2024-04-22T06:28:37.574 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 7.4 M 2024-04-22T06:28:37.574 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 49 k 2024-04-22T06:28:37.574 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-radosgw x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 11 M 2024-04-22T06:28:37.574 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-test x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 49 M 2024-04-22T06:28:37.575 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-volume noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 273 k 2024-04-22T06:28:37.575 INFO:teuthology.orchestra.run.smithi070.stdout: cephadm noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 763 k 2024-04-22T06:28:37.575 INFO:teuthology.orchestra.run.smithi070.stdout: cephfs-mirror x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 225 k 2024-04-22T06:28:37.575 INFO:teuthology.orchestra.run.smithi070.stdout: cephfs-shell x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 27 k 2024-04-22T06:28:37.575 INFO:teuthology.orchestra.run.smithi070.stdout: cephfs-top noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 18 k 2024-04-22T06:28:37.575 INFO:teuthology.orchestra.run.smithi070.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-22T06:28:37.575 INFO:teuthology.orchestra.run.smithi070.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-22T06:28:37.575 INFO:teuthology.orchestra.run.smithi070.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-22T06:28:37.575 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs-devel x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 32 k 2024-04-22T06:28:37.575 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs2 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 747 k 2024-04-22T06:28:37.575 INFO:teuthology.orchestra.run.smithi070.stdout: librados-devel x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 127 k 2024-04-22T06:28:37.575 INFO:teuthology.orchestra.run.smithi070.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-22T06:28:37.576 INFO:teuthology.orchestra.run.smithi070.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-22T06:28:37.576 INFO:teuthology.orchestra.run.smithi070.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-22T06:28:37.576 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-22T06:28:37.576 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-22T06:28:37.576 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-22T06:28:37.576 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cephfs x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 162 k 2024-04-22T06:28:37.576 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rados x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 320 k 2024-04-22T06:28:37.576 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rbd x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 298 k 2024-04-22T06:28:37.576 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rgw x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 100 k 2024-04-22T06:28:37.576 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-fuse x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 85 k 2024-04-22T06:28:37.576 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-mirror x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 3.1 M 2024-04-22T06:28:37.576 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-nbd x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 171 k 2024-04-22T06:28:37.576 INFO:teuthology.orchestra.run.smithi070.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-22T06:28:37.577 INFO:teuthology.orchestra.run.smithi070.stdout:Upgrading: 2024-04-22T06:28:37.577 INFO:teuthology.orchestra.run.smithi070.stdout: librados2 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 3.4 M 2024-04-22T06:28:37.577 INFO:teuthology.orchestra.run.smithi070.stdout: librbd1 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 3.1 M 2024-04-22T06:28:37.577 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-04-22T06:28:37.577 INFO:teuthology.orchestra.run.smithi070.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-22T06:28:37.577 INFO:teuthology.orchestra.run.smithi070.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-22T06:28:37.577 INFO:teuthology.orchestra.run.smithi070.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-22T06:28:37.577 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-common x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 21 M 2024-04-22T06:28:37.577 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 34 k 2024-04-22T06:28:37.577 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mds x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 2.4 M 2024-04-22T06:28:37.577 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 249 k 2024-04-22T06:28:37.577 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mon x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 4.7 M 2024-04-22T06:28:37.577 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-osd x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 17 M 2024-04-22T06:28:37.578 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 16 k 2024-04-22T06:28:37.578 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-selinux x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 25 k 2024-04-22T06:28:37.578 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-22T06:28:37.578 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-22T06:28:37.578 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-22T06:28:37.578 INFO:teuthology.orchestra.run.smithi070.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-22T06:28:37.578 INFO:teuthology.orchestra.run.smithi070.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-22T06:28:37.578 INFO:teuthology.orchestra.run.smithi070.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-22T06:28:37.578 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-22T06:28:37.578 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-22T06:28:37.578 INFO:teuthology.orchestra.run.smithi070.stdout: libcephsqlite x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 164 k 2024-04-22T06:28:37.578 INFO:teuthology.orchestra.run.smithi070.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-22T06:28:37.579 INFO:teuthology.orchestra.run.smithi070.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-22T06:28:37.579 INFO:teuthology.orchestra.run.smithi070.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-22T06:28:37.579 INFO:teuthology.orchestra.run.smithi070.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-22T06:28:37.579 INFO:teuthology.orchestra.run.smithi070.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-22T06:28:37.579 INFO:teuthology.orchestra.run.smithi070.stdout: libradosstriper1 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 500 k 2024-04-22T06:28:37.579 INFO:teuthology.orchestra.run.smithi070.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-22T06:28:37.579 INFO:teuthology.orchestra.run.smithi070.stdout: librgw2 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 5.4 M 2024-04-22T06:28:37.579 INFO:teuthology.orchestra.run.smithi070.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-22T06:28:37.579 INFO:teuthology.orchestra.run.smithi070.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-22T06:28:37.579 INFO:teuthology.orchestra.run.smithi070.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-22T06:28:37.579 INFO:teuthology.orchestra.run.smithi070.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-22T06:28:37.579 INFO:teuthology.orchestra.run.smithi070.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-22T06:28:37.580 INFO:teuthology.orchestra.run.smithi070.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-22T06:28:37.580 INFO:teuthology.orchestra.run.smithi070.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-22T06:28:37.580 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-22T06:28:37.580 INFO:teuthology.orchestra.run.smithi070.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-22T06:28:37.580 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-22T06:28:37.580 INFO:teuthology.orchestra.run.smithi070.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-22T06:28:37.580 INFO:teuthology.orchestra.run.smithi070.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-22T06:28:37.580 INFO:teuthology.orchestra.run.smithi070.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-22T06:28:37.580 INFO:teuthology.orchestra.run.smithi070.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-22T06:28:37.580 INFO:teuthology.orchestra.run.smithi070.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-22T06:28:37.580 INFO:teuthology.orchestra.run.smithi070.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-22T06:28:37.580 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-22T06:28:37.581 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 45 k 2024-04-22T06:28:37.581 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-common x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 135 k 2024-04-22T06:28:37.581 INFO:teuthology.orchestra.run.smithi070.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-22T06:28:37.581 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-22T06:28:37.581 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-22T06:28:37.581 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-22T06:28:37.581 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-22T06:28:37.581 INFO:teuthology.orchestra.run.smithi070.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-22T06:28:37.581 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-22T06:28:37.581 INFO:teuthology.orchestra.run.smithi070.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-22T06:28:37.581 INFO:teuthology.orchestra.run.smithi070.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-22T06:28:37.581 INFO:teuthology.orchestra.run.smithi070.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-22T06:28:37.581 INFO:teuthology.orchestra.run.smithi070.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-22T06:28:37.582 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-22T06:28:37.582 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-22T06:28:37.582 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-22T06:28:37.582 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-22T06:28:37.582 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-22T06:28:37.582 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-22T06:28:37.582 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-22T06:28:37.582 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-22T06:28:37.582 INFO:teuthology.orchestra.run.smithi070.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-22T06:28:37.582 INFO:teuthology.orchestra.run.smithi070.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-22T06:28:37.582 INFO:teuthology.orchestra.run.smithi070.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-22T06:28:37.582 INFO:teuthology.orchestra.run.smithi070.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-22T06:28:37.582 INFO:teuthology.orchestra.run.smithi070.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-22T06:28:37.583 INFO:teuthology.orchestra.run.smithi070.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-22T06:28:37.583 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-22T06:28:37.583 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-22T06:28:37.583 INFO:teuthology.orchestra.run.smithi070.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-22T06:28:37.583 INFO:teuthology.orchestra.run.smithi070.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-22T06:28:37.583 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-22T06:28:37.583 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-22T06:28:37.583 INFO:teuthology.orchestra.run.smithi070.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-22T06:28:37.583 INFO:teuthology.orchestra.run.smithi070.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-22T06:28:37.583 INFO:teuthology.orchestra.run.smithi070.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-22T06:28:37.583 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-22T06:28:37.583 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-22T06:28:37.583 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-22T06:28:37.584 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-22T06:28:37.584 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-22T06:28:37.584 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-22T06:28:37.584 INFO:teuthology.orchestra.run.smithi070.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-22T06:28:37.584 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-22T06:28:37.584 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-22T06:28:37.584 INFO:teuthology.orchestra.run.smithi070.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-22T06:28:37.584 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-22T06:28:37.584 INFO:teuthology.orchestra.run.smithi070.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-22T06:28:37.584 INFO:teuthology.orchestra.run.smithi070.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-22T06:28:37.584 INFO:teuthology.orchestra.run.smithi070.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-22T06:28:37.584 INFO:teuthology.orchestra.run.smithi070.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-22T06:28:37.584 INFO:teuthology.orchestra.run.smithi070.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-22T06:28:37.584 INFO:teuthology.orchestra.run.smithi070.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-22T06:28:37.585 INFO:teuthology.orchestra.run.smithi070.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-22T06:28:37.585 INFO:teuthology.orchestra.run.smithi070.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-22T06:28:37.585 INFO:teuthology.orchestra.run.smithi070.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-22T06:28:37.585 INFO:teuthology.orchestra.run.smithi070.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-22T06:28:37.585 INFO:teuthology.orchestra.run.smithi070.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-22T06:28:37.585 INFO:teuthology.orchestra.run.smithi070.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-22T06:28:37.585 INFO:teuthology.orchestra.run.smithi070.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-22T06:28:37.585 INFO:teuthology.orchestra.run.smithi070.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-22T06:28:37.585 INFO:teuthology.orchestra.run.smithi070.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-22T06:28:37.585 INFO:teuthology.orchestra.run.smithi070.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-22T06:28:37.585 INFO:teuthology.orchestra.run.smithi070.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-22T06:28:37.585 INFO:teuthology.orchestra.run.smithi070.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-22T06:28:37.585 INFO:teuthology.orchestra.run.smithi070.stdout:Installing weak dependencies: 2024-04-22T06:28:37.585 INFO:teuthology.orchestra.run.smithi070.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-22T06:28:37.586 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:28:37.586 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-04-22T06:28:37.586 INFO:teuthology.orchestra.run.smithi070.stdout:======================================================================================= 2024-04-22T06:28:37.586 INFO:teuthology.orchestra.run.smithi070.stdout:Install 148 Packages 2024-04-22T06:28:37.586 INFO:teuthology.orchestra.run.smithi070.stdout:Upgrade 2 Packages 2024-04-22T06:28:37.586 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:28:37.586 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 218 M 2024-04-22T06:28:37.587 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-04-22T06:28:37.739 INFO:teuthology.orchestra.run.smithi044.stdout:(71/150): openblas-openmp-0.3.21-2.el9.x86_64.r 1.8 MB/s | 4.7 MB 00:02 2024-04-22T06:28:37.922 INFO:teuthology.orchestra.run.smithi044.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 477 kB/s | 87 kB 00:00 2024-04-22T06:28:38.272 INFO:teuthology.orchestra.run.smithi044.stdout:(73/150): protobuf-3.14.0-13.el9.x86_64.rpm 733 kB/s | 1.0 MB 00:01 2024-04-22T06:28:38.623 INFO:teuthology.orchestra.run.smithi044.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 698 kB/s | 244 kB 00:00 2024-04-22T06:28:38.695 INFO:teuthology.orchestra.run.smithi070.stdout:(1/150): ceph-19.0.0-3156.g86f7587a.el9.x86_64. 31 kB/s | 6.6 kB 00:00 2024-04-22T06:28:38.973 INFO:teuthology.orchestra.run.smithi044.stdout:(75/150): python3-jinja2-2.11.3-5.el9.noarch.rp 708 kB/s | 248 kB 00:00 2024-04-22T06:28:39.073 INFO:teuthology.orchestra.run.smithi044.stdout:(76/150): python3-jsonpatch-1.21-16.el9.noarch. 257 kB/s | 26 kB 00:00 2024-04-22T06:28:39.174 INFO:teuthology.orchestra.run.smithi044.stdout:(77/150): python3-jsonpointer-2.0-4.el9.noarch. 191 kB/s | 19 kB 00:00 2024-04-22T06:28:39.229 INFO:teuthology.orchestra.run.smithi070.stdout:(2/150): ceph-fuse-19.0.0-3156.g86f7587a.el9.x8 1.6 MB/s | 890 kB 00:00 2024-04-22T06:28:39.329 INFO:teuthology.orchestra.run.smithi070.stdout:(3/150): ceph-immutable-object-cache-19.0.0-315 1.4 MB/s | 145 kB 00:00 2024-04-22T06:28:39.440 INFO:teuthology.orchestra.run.smithi044.stdout:(78/150): python3-mako-1.1.4-6.el9.noarch.rpm 646 kB/s | 172 kB 00:00 2024-04-22T06:28:39.541 INFO:teuthology.orchestra.run.smithi044.stdout:(79/150): python3-markupsafe-1.1.1-12.el9.x86_6 348 kB/s | 35 kB 00:00 2024-04-22T06:28:40.030 INFO:teuthology.orchestra.run.smithi070.stdout:(4/150): ceph-mds-19.0.0-3156.g86f7587a.el9.x86 3.4 MB/s | 2.4 MB 00:00 2024-04-22T06:28:40.421 INFO:teuthology.orchestra.run.smithi070.stdout:(5/150): ceph-common-19.0.0-3156.g86f7587a.el9. 11 MB/s | 21 MB 00:01 2024-04-22T06:28:40.505 INFO:teuthology.orchestra.run.smithi070.stdout:(6/150): ceph-base-19.0.0-3156.g86f7587a.el9.x8 2.7 MB/s | 5.4 MB 00:02 2024-04-22T06:28:40.547 INFO:teuthology.orchestra.run.smithi070.stdout:(7/150): ceph-mgr-19.0.0-3156.g86f7587a.el9.x86 2.9 MB/s | 1.5 MB 00:00 2024-04-22T06:28:40.706 INFO:teuthology.orchestra.run.smithi070.stdout:(8/150): ceph-mon-19.0.0-3156.g86f7587a.el9.x86 17 MB/s | 4.7 MB 00:00 2024-04-22T06:28:40.739 INFO:teuthology.orchestra.run.smithi070.stdout:(9/150): ceph-selinux-19.0.0-3156.g86f7587a.el9 762 kB/s | 25 kB 00:00 2024-04-22T06:28:42.731 INFO:teuthology.orchestra.run.smithi070.stdout:(10/150): ceph-radosgw-19.0.0-3156.g86f7587a.el 4.8 MB/s | 11 MB 00:02 2024-04-22T06:28:42.814 INFO:teuthology.orchestra.run.smithi070.stdout:(11/150): cephfs-mirror-19.0.0-3156.g86f7587a.e 2.6 MB/s | 225 kB 00:00 2024-04-22T06:28:42.906 INFO:teuthology.orchestra.run.smithi070.stdout:(12/150): cephfs-shell-19.0.0-3156.g86f7587a.el 295 kB/s | 27 kB 00:00 2024-04-22T06:28:42.998 INFO:teuthology.orchestra.run.smithi070.stdout:(13/150): libcephfs-devel-19.0.0-3156.g86f7587a 346 kB/s | 32 kB 00:00 2024-04-22T06:28:43.498 INFO:teuthology.orchestra.run.smithi070.stdout:(14/150): ceph-test-19.0.0-3156.g86f7587a.el9.x 18 MB/s | 49 MB 00:02 2024-04-22T06:28:43.532 INFO:teuthology.orchestra.run.smithi070.stdout:(15/150): libcephfs2-19.0.0-3156.g86f7587a.el9. 1.4 MB/s | 747 kB 00:00 2024-04-22T06:28:43.557 INFO:teuthology.orchestra.run.smithi070.stdout:(16/150): libcephsqlite-19.0.0-3156.g86f7587a.e 2.7 MB/s | 164 kB 00:00 2024-04-22T06:28:43.599 INFO:teuthology.orchestra.run.smithi070.stdout:(17/150): librados-devel-19.0.0-3156.g86f7587a. 1.9 MB/s | 127 kB 00:00 2024-04-22T06:28:43.649 INFO:teuthology.orchestra.run.smithi070.stdout:(18/150): libradosstriper1-19.0.0-3156.g86f7587 5.3 MB/s | 500 kB 00:00 2024-04-22T06:28:43.708 INFO:teuthology.orchestra.run.smithi070.stdout:(19/150): python3-ceph-argparse-19.0.0-3156.g86 774 kB/s | 45 kB 00:00 2024-04-22T06:28:43.767 INFO:teuthology.orchestra.run.smithi070.stdout:(20/150): python3-ceph-common-19.0.0-3156.g86f7 2.3 MB/s | 135 kB 00:00 2024-04-22T06:28:43.825 INFO:teuthology.orchestra.run.smithi070.stdout:(21/150): python3-cephfs-19.0.0-3156.g86f7587a. 2.7 MB/s | 162 kB 00:00 2024-04-22T06:28:43.892 INFO:teuthology.orchestra.run.smithi070.stdout:(22/150): python3-rados-19.0.0-3156.g86f7587a.e 4.7 MB/s | 320 kB 00:00 2024-04-22T06:28:43.942 INFO:teuthology.orchestra.run.smithi070.stdout:(23/150): python3-rbd-19.0.0-3156.g86f7587a.el9 5.8 MB/s | 298 kB 00:00 2024-04-22T06:28:43.993 INFO:teuthology.orchestra.run.smithi070.stdout:(24/150): python3-rgw-19.0.0-3156.g86f7587a.el9 1.9 MB/s | 100 kB 00:00 2024-04-22T06:28:44.051 INFO:teuthology.orchestra.run.smithi070.stdout:(25/150): rbd-fuse-19.0.0-3156.g86f7587a.el9.x8 1.4 MB/s | 85 kB 00:00 2024-04-22T06:28:44.293 INFO:teuthology.orchestra.run.smithi070.stdout:(26/150): rbd-mirror-19.0.0-3156.g86f7587a.el9. 13 MB/s | 3.1 MB 00:00 2024-04-22T06:28:44.344 INFO:teuthology.orchestra.run.smithi070.stdout:(27/150): rbd-nbd-19.0.0-3156.g86f7587a.el9.x86 3.4 MB/s | 171 kB 00:00 2024-04-22T06:28:44.419 INFO:teuthology.orchestra.run.smithi070.stdout:(28/150): librgw2-19.0.0-3156.g86f7587a.el9.x86 6.6 MB/s | 5.4 MB 00:00 2024-04-22T06:28:44.445 INFO:teuthology.orchestra.run.smithi070.stdout:(29/150): ceph-grafana-dashboards-19.0.0-3156.g 339 kB/s | 34 kB 00:00 2024-04-22T06:28:44.478 INFO:teuthology.orchestra.run.smithi070.stdout:(30/150): ceph-mgr-cephadm-19.0.0-3156.g86f7587 2.4 MB/s | 142 kB 00:00 2024-04-22T06:28:44.482 INFO:teuthology.orchestra.run.smithi044.stdout:(80/150): postgresql-server-13.14-1.el9.x86_64. 755 kB/s | 5.8 MB 00:07 2024-04-22T06:28:44.695 INFO:teuthology.orchestra.run.smithi070.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3156.g86f75 14 MB/s | 3.5 MB 00:00 2024-04-22T06:28:44.824 INFO:teuthology.orchestra.run.smithi044.stdout:(81/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 734 kB/s | 250 kB 00:00 2024-04-22T06:28:44.879 INFO:teuthology.orchestra.run.smithi070.stdout:(32/150): ceph-osd-19.0.0-3156.g86f7587a.el9.x8 3.8 MB/s | 17 MB 00:04 2024-04-22T06:28:44.904 INFO:teuthology.orchestra.run.smithi070.stdout:(33/150): ceph-mgr-modules-core-19.0.0-3156.g86 1.2 MB/s | 249 kB 00:00 2024-04-22T06:28:44.938 INFO:teuthology.orchestra.run.smithi070.stdout:(34/150): ceph-mgr-rook-19.0.0-3156.g86f7587a.e 843 kB/s | 49 kB 00:00 2024-04-22T06:28:44.955 INFO:teuthology.orchestra.run.smithi070.stdout:(35/150): ceph-prometheus-alerts-19.0.0-3156.g8 328 kB/s | 16 kB 00:00 2024-04-22T06:28:45.005 INFO:teuthology.orchestra.run.smithi070.stdout:(36/150): ceph-volume-19.0.0-3156.g86f7587a.el9 4.0 MB/s | 273 kB 00:00 2024-04-22T06:28:45.039 INFO:teuthology.orchestra.run.smithi070.stdout:(37/150): cephadm-19.0.0-3156.g86f7587a.el9.noa 8.9 MB/s | 763 kB 00:00 2024-04-22T06:28:45.064 INFO:teuthology.orchestra.run.smithi070.stdout:(38/150): cephfs-top-19.0.0-3156.g86f7587a.el9. 311 kB/s | 18 kB 00:00 2024-04-22T06:28:45.156 INFO:teuthology.orchestra.run.smithi070.stdout:(39/150): fuse-2.9.9-15.el9.x86_64.rpm 688 kB/s | 80 kB 00:00 2024-04-22T06:28:45.166 INFO:teuthology.orchestra.run.smithi044.stdout:(82/150): python3-oauthlib-3.1.1-5.el9.noarch.r 650 kB/s | 222 kB 00:00 2024-04-22T06:28:45.239 INFO:teuthology.orchestra.run.smithi070.stdout:(40/150): libgfortran-11.4.1-3.el9.x86_64.rpm 4.5 MB/s | 803 kB 00:00 2024-04-22T06:28:45.265 INFO:teuthology.orchestra.run.smithi070.stdout:(41/150): libquadmath-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 195 kB 00:00 2024-04-22T06:28:45.298 INFO:teuthology.orchestra.run.smithi070.stdout:(42/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-04-22T06:28:45.340 INFO:teuthology.orchestra.run.smithi070.stdout:(43/150): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-04-22T06:28:45.349 INFO:teuthology.orchestra.run.smithi044.stdout:(83/150): python3-packaging-20.9-5.el9.noarch.r 421 kB/s | 77 kB 00:00 2024-04-22T06:28:45.365 INFO:teuthology.orchestra.run.smithi070.stdout:(44/150): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-22T06:28:45.391 INFO:teuthology.orchestra.run.smithi070.stdout:(45/150): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-22T06:28:45.416 INFO:teuthology.orchestra.run.smithi070.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-22T06:28:45.450 INFO:teuthology.orchestra.run.smithi070.stdout:(47/150): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-04-22T06:28:45.458 INFO:teuthology.orchestra.run.smithi044.stdout:(84/150): python3-prettytable-0.7.2-27.el9.noar 389 kB/s | 42 kB 00:00 2024-04-22T06:28:45.541 INFO:teuthology.orchestra.run.smithi070.stdout:(48/150): ceph-mgr-diskprediction-local-19.0.0- 7.0 MB/s | 7.4 MB 00:01 2024-04-22T06:28:45.575 INFO:teuthology.orchestra.run.smithi070.stdout:(49/150): automake-1.16.2-8.el9.noarch.rpm 4.2 MB/s | 689 kB 00:00 2024-04-22T06:28:45.617 INFO:teuthology.orchestra.run.smithi070.stdout:(50/150): bison-3.7.4-5.el9.x86_64.rpm 5.5 MB/s | 943 kB 00:00 2024-04-22T06:28:45.642 INFO:teuthology.orchestra.run.smithi070.stdout:(51/150): elfutils-libelf-devel-0.190-2.el9.x86 321 kB/s | 21 kB 00:00 2024-04-22T06:28:45.668 INFO:teuthology.orchestra.run.smithi070.stdout:(52/150): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-22T06:28:45.693 INFO:teuthology.orchestra.run.smithi070.stdout:(53/150): boost-program-options-1.75.0-8.el9.x8 708 kB/s | 107 kB 00:00 2024-04-22T06:28:45.718 INFO:teuthology.orchestra.run.smithi070.stdout:(54/150): flexiblas-3.0.4-8.el9.x86_64.rpm 433 kB/s | 33 kB 00:00 2024-04-22T06:28:45.743 INFO:teuthology.orchestra.run.smithi070.stdout:(55/150): flexiblas-openblas-openmp-3.0.4-8.el9 362 kB/s | 18 kB 00:00 2024-04-22T06:28:45.769 INFO:teuthology.orchestra.run.smithi070.stdout:(56/150): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-04-22T06:28:45.794 INFO:teuthology.orchestra.run.smithi070.stdout:(57/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-22T06:28:45.819 INFO:teuthology.orchestra.run.smithi070.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 904 kB/s | 45 kB 00:00 2024-04-22T06:28:45.853 INFO:teuthology.orchestra.run.smithi070.stdout:(59/150): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-22T06:28:45.892 INFO:teuthology.orchestra.run.smithi044.stdout:(85/150): python3-protobuf-3.14.0-13.el9.noarch 622 kB/s | 269 kB 00:00 2024-04-22T06:28:45.911 INFO:teuthology.orchestra.run.smithi070.stdout:(60/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-04-22T06:28:45.937 INFO:teuthology.orchestra.run.smithi070.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 305 kB/s | 25 kB 00:00 2024-04-22T06:28:45.962 INFO:teuthology.orchestra.run.smithi070.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-04-22T06:28:45.990 INFO:teuthology.orchestra.run.smithi070.stdout:(63/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-22T06:28:46.012 INFO:teuthology.orchestra.run.smithi070.stdout:(64/150): libtool-2.4.6-45.el9.x86_64.rpm 2.9 MB/s | 582 kB 00:00 2024-04-22T06:28:46.029 INFO:teuthology.orchestra.run.smithi070.stdout:(65/150): openblas-0.3.21-2.el9.x86_64.rpm 836 kB/s | 35 kB 00:00 2024-04-22T06:28:46.055 INFO:teuthology.orchestra.run.smithi070.stdout:(66/150): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-04-22T06:28:46.117 INFO:teuthology.orchestra.run.smithi044.stdout:(86/150): python3-babel-2.9.1-2.el9.noarch.rpm 745 kB/s | 6.0 MB 00:08 2024-04-22T06:28:46.122 INFO:teuthology.orchestra.run.smithi070.stdout:(67/150): postgresql-13.14-1.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-04-22T06:28:46.197 INFO:teuthology.orchestra.run.smithi070.stdout:(68/150): openblas-openmp-0.3.21-2.el9.x86_64.r 26 MB/s | 4.7 MB 00:00 2024-04-22T06:28:46.200 INFO:teuthology.orchestra.run.smithi044.stdout:(87/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 514 kB/s | 159 kB 00:00 2024-04-22T06:28:46.256 INFO:teuthology.orchestra.run.smithi070.stdout:(69/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 18 MB/s | 4.1 MB 00:00 2024-04-22T06:28:46.281 INFO:teuthology.orchestra.run.smithi070.stdout:(70/150): postgresql-private-libs-13.14-1.el9.x 1.6 MB/s | 133 kB 00:00 2024-04-22T06:28:46.284 INFO:teuthology.orchestra.run.smithi044.stdout:(88/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 774 kB/s | 5.1 MB 00:06 2024-04-22T06:28:46.301 INFO:teuthology.orchestra.run.smithi044.stdout:(89/150): python3-pytz-2021.1-5.el9.noarch.rpm 509 kB/s | 51 kB 00:00 2024-04-22T06:28:46.314 INFO:teuthology.orchestra.run.smithi070.stdout:(71/150): postgresql-contrib-13.14-1.el9.x86_64 4.6 MB/s | 911 kB 00:00 2024-04-22T06:28:46.348 INFO:teuthology.orchestra.run.smithi070.stdout:(72/150): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-04-22T06:28:46.373 INFO:teuthology.orchestra.run.smithi070.stdout:(73/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-22T06:28:46.385 INFO:teuthology.orchestra.run.smithi044.stdout:(90/150): python3-requests-oauthlib-1.3.0-12.el 536 kB/s | 54 kB 00:00 2024-04-22T06:28:46.415 INFO:teuthology.orchestra.run.smithi070.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 5.7 MB/s | 244 kB 00:00 2024-04-22T06:28:46.477 INFO:teuthology.orchestra.run.smithi044.stdout:(91/150): python3-toml-0.10.2-6.el9.noarch.rpm 454 kB/s | 42 kB 00:00 2024-04-22T06:28:46.490 INFO:teuthology.orchestra.run.smithi070.stdout:(75/150): postgresql-server-13.14-1.el9.x86_64. 25 MB/s | 5.8 MB 00:00 2024-04-22T06:28:46.516 INFO:teuthology.orchestra.run.smithi070.stdout:(76/150): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-04-22T06:28:46.541 INFO:teuthology.orchestra.run.smithi070.stdout:(77/150): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-04-22T06:28:46.566 INFO:teuthology.orchestra.run.smithi070.stdout:(78/150): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-04-22T06:28:46.577 INFO:teuthology.orchestra.run.smithi044.stdout:(92/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 443 kB/s | 44 kB 00:00 2024-04-22T06:28:46.591 INFO:teuthology.orchestra.run.smithi070.stdout:(79/150): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-22T06:28:46.602 INFO:teuthology.orchestra.run.smithi044.stdout:(93/150): python3-pyasn1-modules-0.4.8-6.el9.no 575 kB/s | 279 kB 00:00 2024-04-22T06:28:46.617 INFO:teuthology.orchestra.run.smithi070.stdout:(80/150): python3-markupsafe-1.1.1-12.el9.x86_6 692 kB/s | 35 kB 00:00 2024-04-22T06:28:46.709 INFO:teuthology.orchestra.run.smithi070.stdout:(81/150): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-04-22T06:28:46.711 INFO:teuthology.orchestra.run.smithi044.stdout:(94/150): qatlib-service-24.02.0-1.el9.x86_64.r 332 kB/s | 36 kB 00:00 2024-04-22T06:28:46.734 INFO:teuthology.orchestra.run.smithi070.stdout:(82/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-04-22T06:28:46.759 INFO:teuthology.orchestra.run.smithi070.stdout:(83/150): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-04-22T06:28:46.785 INFO:teuthology.orchestra.run.smithi070.stdout:(84/150): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-22T06:28:46.810 INFO:teuthology.orchestra.run.smithi070.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 841 kB/s | 42 kB 00:00 2024-04-22T06:28:46.819 INFO:teuthology.orchestra.run.smithi044.stdout:(95/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 437 kB/s | 47 kB 00:00 2024-04-22T06:28:46.835 INFO:teuthology.orchestra.run.smithi070.stdout:(86/150): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-22T06:28:46.860 INFO:teuthology.orchestra.run.smithi070.stdout:(87/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-22T06:28:46.894 INFO:teuthology.orchestra.run.smithi070.stdout:(88/150): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-04-22T06:28:46.919 INFO:teuthology.orchestra.run.smithi070.stdout:(89/150): python3-pytz-2021.1-5.el9.noarch.rpm 872 kB/s | 51 kB 00:00 2024-04-22T06:28:46.928 INFO:teuthology.orchestra.run.smithi044.stdout:(96/150): qatlib-24.02.0-1.el9.x86_64.rpm 628 kB/s | 221 kB 00:00 2024-04-22T06:28:46.945 INFO:teuthology.orchestra.run.smithi070.stdout:(90/150): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-22T06:28:47.020 INFO:teuthology.orchestra.run.smithi070.stdout:(91/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-04-22T06:28:47.028 INFO:teuthology.orchestra.run.smithi044.stdout:(97/150): uuid-1.6.2-55.el9.x86_64.rpm 579 kB/s | 58 kB 00:00 2024-04-22T06:28:47.045 INFO:teuthology.orchestra.run.smithi070.stdout:(92/150): python3-toml-0.10.2-6.el9.noarch.rpm 416 kB/s | 42 kB 00:00 2024-04-22T06:28:47.070 INFO:teuthology.orchestra.run.smithi070.stdout:(93/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 882 kB/s | 44 kB 00:00 2024-04-22T06:28:47.095 INFO:teuthology.orchestra.run.smithi070.stdout:(94/150): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-22T06:28:47.123 INFO:teuthology.orchestra.run.smithi070.stdout:(95/150): qatlib-service-24.02.0-1.el9.x86_64.r 687 kB/s | 36 kB 00:00 2024-04-22T06:28:47.140 INFO:teuthology.orchestra.run.smithi070.stdout:(96/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-04-22T06:28:47.165 INFO:teuthology.orchestra.run.smithi070.stdout:(97/150): socat-1.7.4.1-5.el9.x86_64.rpm 7.1 MB/s | 305 kB 00:00 2024-04-22T06:28:47.182 INFO:teuthology.orchestra.run.smithi070.stdout:(98/150): uuid-1.6.2-55.el9.x86_64.rpm 1.3 MB/s | 58 kB 00:00 2024-04-22T06:28:47.207 INFO:teuthology.orchestra.run.smithi070.stdout:(99/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.2 MB/s | 94 kB 00:00 2024-04-22T06:28:47.212 INFO:teuthology.orchestra.run.smithi044.stdout:(98/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 511 kB/s | 94 kB 00:00 2024-04-22T06:28:47.233 INFO:teuthology.orchestra.run.smithi070.stdout:(100/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-22T06:28:47.258 INFO:teuthology.orchestra.run.smithi070.stdout:(101/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 891 kB/s | 45 kB 00:00 2024-04-22T06:28:47.321 INFO:teuthology.orchestra.run.smithi044.stdout:(99/150): socat-1.7.4.1-5.el9.x86_64.rpm 608 kB/s | 305 kB 00:00 2024-04-22T06:28:47.341 INFO:teuthology.orchestra.run.smithi070.stdout:(102/150): lua-devel-5.4.4-4.el9.x86_64.rpm 206 kB/s | 22 kB 00:00 2024-04-22T06:28:47.392 INFO:teuthology.orchestra.run.smithi070.stdout:(103/150): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-04-22T06:28:47.480 INFO:teuthology.orchestra.run.smithi044.stdout:(100/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 238 kB/s | 64 kB 00:00 2024-04-22T06:28:47.505 INFO:teuthology.orchestra.run.smithi044.stdout:(101/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 244 kB/s | 45 kB 00:00 2024-04-22T06:28:47.630 INFO:teuthology.orchestra.run.smithi044.stdout:(102/150): lua-devel-5.4.4-4.el9.x86_64.rpm 149 kB/s | 22 kB 00:00 2024-04-22T06:28:47.667 INFO:teuthology.orchestra.run.smithi070.stdout:(104/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-04-22T06:28:47.784 INFO:teuthology.orchestra.run.smithi070.stdout:(105/150): gperftools-libs-2.9.1-2.el9.x86_64.r 788 kB/s | 309 kB 00:00 2024-04-22T06:28:47.801 INFO:teuthology.orchestra.run.smithi070.stdout:(106/150): grpc-data-1.46.7-10.el9.noarch.rpm 146 kB/s | 19 kB 00:00 2024-04-22T06:28:47.843 INFO:teuthology.orchestra.run.smithi070.stdout:(107/150): protobuf-compiler-3.14.0-13.el9.x86_ 1.4 MB/s | 863 kB 00:00 2024-04-22T06:28:47.860 INFO:teuthology.orchestra.run.smithi070.stdout:(108/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 444 kB/s | 26 kB 00:00 2024-04-22T06:28:47.877 INFO:teuthology.orchestra.run.smithi070.stdout:(109/150): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-04-22T06:28:47.902 INFO:teuthology.orchestra.run.smithi070.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-04-22T06:28:47.927 INFO:teuthology.orchestra.run.smithi070.stdout:(111/150): luarocks-3.9.2-1.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-04-22T06:28:47.969 INFO:teuthology.orchestra.run.smithi070.stdout:(112/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 12 MB/s | 838 kB 00:00 2024-04-22T06:28:48.014 INFO:teuthology.orchestra.run.smithi044.stdout:(103/150): protobuf-compiler-3.14.0-13.el9.x86_ 1.7 MB/s | 863 kB 00:00 2024-04-22T06:28:48.095 INFO:teuthology.orchestra.run.smithi070.stdout:(113/150): libarrow-9.0.0-10.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-04-22T06:28:48.120 INFO:teuthology.orchestra.run.smithi070.stdout:(114/150): python3-asyncssh-2.13.2-5.el9.noarch 2.8 MB/s | 548 kB 00:00 2024-04-22T06:28:48.145 INFO:teuthology.orchestra.run.smithi070.stdout:(115/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 248 kB/s | 43 kB 00:00 2024-04-22T06:28:48.171 INFO:teuthology.orchestra.run.smithi070.stdout:(116/150): python3-cachetools-4.2.4-1.el9.noarc 428 kB/s | 32 kB 00:00 2024-04-22T06:28:48.196 INFO:teuthology.orchestra.run.smithi070.stdout:(117/150): python3-certifi-2021.10.8-2.el9.noar 199 kB/s | 15 kB 00:00 2024-04-22T06:28:48.214 INFO:teuthology.orchestra.run.smithi044.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 939 kB/s | 548 kB 00:00 2024-04-22T06:28:48.221 INFO:teuthology.orchestra.run.smithi070.stdout:(118/150): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-04-22T06:28:48.246 INFO:teuthology.orchestra.run.smithi070.stdout:(119/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-04-22T06:28:48.271 INFO:teuthology.orchestra.run.smithi070.stdout:(120/150): python3-colorama-0.4.4-12.el9.noarch 703 kB/s | 35 kB 00:00 2024-04-22T06:28:48.296 INFO:teuthology.orchestra.run.smithi070.stdout:(121/150): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-04-22T06:28:48.322 INFO:teuthology.orchestra.run.smithi070.stdout:(122/150): python3-google-auth-2.28.2-1.el9.noa 2.6 MB/s | 203 kB 00:00 2024-04-22T06:28:48.347 INFO:teuthology.orchestra.run.smithi070.stdout:(123/150): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-04-22T06:28:48.372 INFO:teuthology.orchestra.run.smithi070.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-04-22T06:28:48.398 INFO:teuthology.orchestra.run.smithi070.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-04-22T06:28:48.423 INFO:teuthology.orchestra.run.smithi070.stdout:(126/150): python3-jaraco-collections-3.0.0-8.e 459 kB/s | 23 kB 00:00 2024-04-22T06:28:48.448 INFO:teuthology.orchestra.run.smithi070.stdout:(127/150): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-04-22T06:28:48.474 INFO:teuthology.orchestra.run.smithi070.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-04-22T06:28:48.481 INFO:teuthology.orchestra.run.smithi044.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 73 kB/s | 19 kB 00:00 2024-04-22T06:28:48.507 INFO:teuthology.orchestra.run.smithi070.stdout:(129/150): python3-kubernetes-26.1.0-2.el9.noar 17 MB/s | 1.0 MB 00:00 2024-04-22T06:28:48.522 INFO:teuthology.orchestra.run.smithi044.stdout:(106/150): gperftools-libs-2.9.1-2.el9.x86_64.r 607 kB/s | 309 kB 00:00 2024-04-22T06:28:48.532 INFO:teuthology.orchestra.run.smithi070.stdout:(130/150): python3-logutils-0.3.5-21.el9.noarch 793 kB/s | 46 kB 00:00 2024-04-22T06:28:48.558 INFO:teuthology.orchestra.run.smithi070.stdout:(131/150): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-04-22T06:28:48.583 INFO:teuthology.orchestra.run.smithi070.stdout:(132/150): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-04-22T06:28:48.608 INFO:teuthology.orchestra.run.smithi070.stdout:(133/150): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-04-22T06:28:48.634 INFO:teuthology.orchestra.run.smithi070.stdout:(134/150): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-04-22T06:28:48.679 INFO:teuthology.orchestra.run.smithi070.stdout:(135/150): python3-grpcio-1.46.7-10.el9.x86_64. 5.0 MB/s | 2.0 MB 00:00 2024-04-22T06:28:48.701 INFO:teuthology.orchestra.run.smithi070.stdout:(136/150): python3-pyperclip-1.8.2-2.el9.noarch 423 kB/s | 28 kB 00:00 2024-04-22T06:28:48.726 INFO:teuthology.orchestra.run.smithi070.stdout:(137/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 766 kB/s | 90 kB 00:00 2024-04-22T06:28:48.751 INFO:teuthology.orchestra.run.smithi070.stdout:(138/150): python3-repoze-lru-0.7-16.el9.noarch 409 kB/s | 31 kB 00:00 2024-04-22T06:28:48.777 INFO:teuthology.orchestra.run.smithi070.stdout:(139/150): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-04-22T06:28:48.790 INFO:teuthology.orchestra.run.smithi044.stdout:(107/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 97 kB/s | 26 kB 00:00 2024-04-22T06:28:48.802 INFO:teuthology.orchestra.run.smithi070.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 715 kB/s | 36 kB 00:00 2024-04-22T06:28:48.827 INFO:teuthology.orchestra.run.smithi070.stdout:(141/150): python3-rsa-4.9-2.el9.noarch.rpm 784 kB/s | 59 kB 00:00 2024-04-22T06:28:48.852 INFO:teuthology.orchestra.run.smithi070.stdout:(142/150): python3-typing-extensions-4.4.0-2.el 675 kB/s | 51 kB 00:00 2024-04-22T06:28:48.878 INFO:teuthology.orchestra.run.smithi070.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-04-22T06:28:48.903 INFO:teuthology.orchestra.run.smithi070.stdout:(144/150): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-22T06:28:48.919 INFO:teuthology.orchestra.run.smithi070.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 479 kB/s | 20 kB 00:00 2024-04-22T06:28:48.945 INFO:teuthology.orchestra.run.smithi070.stdout:(146/150): python3-werkzeug-2.0.3-3.el9.1.noarc 4.5 MB/s | 427 kB 00:00 2024-04-22T06:28:48.970 INFO:teuthology.orchestra.run.smithi070.stdout:(147/150): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-04-22T06:28:49.012 INFO:teuthology.orchestra.run.smithi070.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-04-22T06:28:49.140 INFO:teuthology.orchestra.run.smithi044.stdout:(108/150): liboath-2.6.7-2.el9.x86_64.rpm 139 kB/s | 49 kB 00:00 2024-04-22T06:28:49.482 INFO:teuthology.orchestra.run.smithi044.stdout:(109/150): libunwind-1.6.2-1.el9.x86_64.rpm 197 kB/s | 67 kB 00:00 2024-04-22T06:28:49.529 INFO:teuthology.orchestra.run.smithi070.stdout:(149/150): librados2-19.0.0-3156.g86f7587a.el9. 5.8 MB/s | 3.4 MB 00:00 2024-04-22T06:28:49.696 INFO:teuthology.orchestra.run.smithi070.stdout:(150/150): librbd1-19.0.0-3156.g86f7587a.el9.x8 4.3 MB/s | 3.1 MB 00:00 2024-04-22T06:28:49.698 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-04-22T06:28:49.699 INFO:teuthology.orchestra.run.smithi070.stdout:Total 18 MB/s | 218 MB 00:12 2024-04-22T06:28:49.799 INFO:teuthology.orchestra.run.smithi044.stdout:(110/150): libarrow-9.0.0-10.el9.x86_64.rpm 3.4 MB/s | 4.4 MB 00:01 2024-04-22T06:28:49.907 INFO:teuthology.orchestra.run.smithi044.stdout:(111/150): luarocks-3.9.2-1.el9.noarch.rpm 355 kB/s | 151 kB 00:00 2024-04-22T06:28:50.483 INFO:teuthology.orchestra.run.smithi044.stdout:(112/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.2 MB/s | 838 kB 00:00 2024-04-22T06:28:50.516 INFO:teuthology.orchestra.run.smithi044.stdout:(113/150): python3-asyncssh-2.13.2-5.el9.noarch 901 kB/s | 548 kB 00:00 2024-04-22T06:28:50.723 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-04-22T06:28:50.783 INFO:teuthology.orchestra.run.smithi044.stdout:(114/150): python3-cachetools-4.2.4-1.el9.noarc 121 kB/s | 32 kB 00:00 2024-04-22T06:28:50.825 INFO:teuthology.orchestra.run.smithi044.stdout:(115/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 127 kB/s | 43 kB 00:00 2024-04-22T06:28:50.849 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-04-22T06:28:50.849 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-04-22T06:28:51.050 INFO:teuthology.orchestra.run.smithi044.stdout:(116/150): python3-certifi-2021.10.8-2.el9.noar 56 kB/s | 15 kB 00:00 2024-04-22T06:28:51.250 INFO:teuthology.orchestra.run.smithi044.stdout:(117/150): python3-cheroot-8.6.0-4.el9.noarch.r 404 kB/s | 172 kB 00:00 2024-04-22T06:28:51.601 INFO:teuthology.orchestra.run.smithi044.stdout:(118/150): python3-cherrypy-18.6.1-2.el9.noarch 651 kB/s | 358 kB 00:00 2024-04-22T06:28:51.768 INFO:teuthology.orchestra.run.smithi044.stdout:(119/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 586 kB/s | 303 kB 00:00 2024-04-22T06:28:51.868 INFO:teuthology.orchestra.run.smithi044.stdout:(120/150): python3-colorama-0.4.4-12.el9.noarch 132 kB/s | 35 kB 00:00 2024-04-22T06:28:52.293 INFO:teuthology.orchestra.run.smithi044.stdout:(121/150): python3-google-auth-2.28.2-1.el9.noa 386 kB/s | 203 kB 00:00 2024-04-22T06:28:52.557 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-04-22T06:28:52.559 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-04-22T06:28:52.702 INFO:teuthology.orchestra.run.smithi044.stdout:(122/150): python3-grpcio-1.46.7-10.el9.x86_64. 2.4 MB/s | 2.0 MB 00:00 2024-04-22T06:28:52.727 INFO:teuthology.orchestra.run.smithi044.stdout:(123/150): python3-grpcio-tools-1.46.7-10.el9.x 333 kB/s | 144 kB 00:00 2024-04-22T06:28:52.886 INFO:teuthology.orchestra.run.smithi044.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 58 kB/s | 11 kB 00:00 2024-04-22T06:28:52.995 INFO:teuthology.orchestra.run.smithi044.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 66 kB/s | 18 kB 00:00 2024-04-22T06:28:53.145 INFO:teuthology.orchestra.run.smithi044.stdout:(126/150): python3-jaraco-collections-3.0.0-8.e 89 kB/s | 23 kB 00:00 2024-04-22T06:28:53.262 INFO:teuthology.orchestra.run.smithi044.stdout:(127/150): python3-jaraco-functools-3.5.0-2.el9 73 kB/s | 19 kB 00:00 2024-04-22T06:28:53.412 INFO:teuthology.orchestra.run.smithi044.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 74 kB/s | 20 kB 00:00 2024-04-22T06:28:53.754 INFO:teuthology.orchestra.run.smithi044.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 135 kB/s | 46 kB 00:00 2024-04-22T06:28:53.946 INFO:teuthology.orchestra.run.smithi044.stdout:(130/150): python3-kubernetes-26.1.0-2.el9.noar 1.5 MB/s | 1.0 MB 00:00 2024-04-22T06:28:54.105 INFO:teuthology.orchestra.run.smithi044.stdout:(131/150): python3-more-itertools-8.12.0-2.el9. 225 kB/s | 79 kB 00:00 2024-04-22T06:28:54.288 INFO:teuthology.orchestra.run.smithi044.stdout:(132/150): python3-natsort-7.1.1-5.el9.noarch.r 169 kB/s | 58 kB 00:00 2024-04-22T06:28:54.555 INFO:teuthology.orchestra.run.smithi044.stdout:(133/150): python3-portend-3.1.0-2.el9.noarch.r 61 kB/s | 16 kB 00:00 2024-04-22T06:28:54.614 INFO:teuthology.orchestra.run.smithi044.stdout:(134/150): python3-pecan-1.4.2-3.el9.noarch.rpm 535 kB/s | 272 kB 00:00 2024-04-22T06:28:54.881 INFO:teuthology.orchestra.run.smithi044.stdout:(135/150): python3-pyperclip-1.8.2-2.el9.noarch 106 kB/s | 28 kB 00:00 2024-04-22T06:28:54.906 INFO:teuthology.orchestra.run.smithi044.stdout:(136/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 257 kB/s | 90 kB 00:00 2024-04-22T06:28:55.190 INFO:teuthology.orchestra.run.smithi044.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 100 kB/s | 31 kB 00:00 2024-04-22T06:28:55.423 INFO:teuthology.orchestra.run.smithi044.stdout:(138/150): python3-routes-2.5.1-5.el9.noarch.rp 365 kB/s | 188 kB 00:00 2024-04-22T06:28:55.540 INFO:teuthology.orchestra.run.smithi044.stdout:(139/150): python3-rsa-4.9-2.el9.noarch.rpm 169 kB/s | 59 kB 00:00 2024-04-22T06:28:55.690 INFO:teuthology.orchestra.run.smithi044.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 134 kB/s | 36 kB 00:00 2024-04-22T06:28:55.882 INFO:teuthology.orchestra.run.smithi044.stdout:(141/150): python3-typing-extensions-4.4.0-2.el 149 kB/s | 51 kB 00:00 2024-04-22T06:28:56.199 INFO:teuthology.orchestra.run.smithi044.stdout:(142/150): python3-webob-1.8.7-6.el9.noarch.rpm 451 kB/s | 230 kB 00:00 2024-04-22T06:28:56.233 INFO:teuthology.orchestra.run.smithi044.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 256 kB/s | 90 kB 00:00 2024-04-22T06:28:56.500 INFO:teuthology.orchestra.run.smithi044.stdout:(144/150): python3-zc-lockfile-2.0-10.el9.noarc 75 kB/s | 20 kB 00:00 2024-04-22T06:28:56.875 INFO:teuthology.orchestra.run.smithi044.stdout:(145/150): python3-werkzeug-2.0.3-3.el9.1.noarc 632 kB/s | 427 kB 00:00 2024-04-22T06:28:57.009 INFO:teuthology.orchestra.run.smithi044.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 377 kB/s | 192 kB 00:00 2024-04-22T06:28:57.501 INFO:teuthology.orchestra.run.smithi044.stdout:(147/150): librados2-19.0.0-3156.g86f7587a.el9. 6.9 MB/s | 3.4 MB 00:00 2024-04-22T06:28:57.626 INFO:teuthology.orchestra.run.smithi044.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 2.1 MB/s | 1.6 MB 00:00 2024-04-22T06:28:57.859 INFO:teuthology.orchestra.run.smithi044.stdout:(149/150): librbd1-19.0.0-3156.g86f7587a.el9.x8 8.8 MB/s | 3.1 MB 00:00 2024-04-22T06:28:58.355 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-04-22T06:28:58.431 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-22T06:28:58.456 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-22T06:28:58.499 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-22T06:28:58.829 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-22T06:28:58.949 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : librados2-2:19.0.0-3156.g86f7587a.el9.x86_64 5/152 2024-04-22T06:28:59.062 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados2-2:19.0.0-3156.g86f7587a.el9.x86_64 5/152 2024-04-22T06:28:59.091 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephfs2-2:19.0.0-3156.g86f7587a.el9.x86_64 6/152 2024-04-22T06:28:59.167 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.g86f7587a.el9.x86_64 6/152 2024-04-22T06:28:59.208 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rados-2:19.0.0-3156.g86f7587a.el9.x86_64 7/152 2024-04-22T06:28:59.238 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-22T06:28:59.263 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-22T06:28:59.422 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-22T06:28:59.469 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-22T06:28:59.497 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephsqlite-2:19.0.0-3156.g86f7587a.el9.x86_64 12/152 2024-04-22T06:28:59.601 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.g86f7587a.el9.x86_64 12/152 2024-04-22T06:28:59.631 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libradosstriper1-2:19.0.0-3156.g86f7587a.el9.x86 13/152 2024-04-22T06:28:59.693 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.g86f7587a.el9.x86 13/152 2024-04-22T06:28:59.715 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-22T06:28:59.738 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-22T06:28:59.906 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-22T06:28:59.940 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-22T06:29:00.057 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-22T06:29:00.155 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-22T06:29:00.207 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-22T06:29:00.250 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-22T06:29:00.325 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-22T06:29:00.365 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-22T06:29:00.409 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-22T06:29:00.441 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-22T06:29:00.469 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-22T06:29:00.540 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-22T06:29:00.592 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-22T06:29:00.622 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-22T06:29:00.665 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-22T06:29:00.742 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-22T06:29:00.776 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ceph-common-2:19.0.0-3156.g86f7587a.el9. 30/152 2024-04-22T06:29:00.819 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.g86f7587a.el 31/152 2024-04-22T06:29:00.848 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cephfs-2:19.0.0-3156.g86f7587a.el9.x86_6 32/152 2024-04-22T06:29:00.920 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-22T06:29:00.963 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-22T06:29:01.039 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-22T06:29:01.184 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-22T06:29:01.242 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-22T06:29:01.783 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-22T06:29:01.872 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-22T06:29:01.906 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-22T06:29:01.941 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-22T06:29:01.979 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librados-devel-2:19.0.0-3156.g86f7587a.el9.x86_6 42/152 2024-04-22T06:29:02.012 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-22T06:29:02.068 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-22T06:29:02.142 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-22T06:29:02.179 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-22T06:29:02.216 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-22T06:29:02.264 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-22T06:29:02.299 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-22T06:29:02.342 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-22T06:29:02.385 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-22T06:29:02.523 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-22T06:29:02.564 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-22T06:29:02.594 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-22T06:29:02.636 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-22T06:29:02.733 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-22T06:29:02.778 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-22T06:29:02.818 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-22T06:29:02.843 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-22T06:29:02.868 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-22T06:29:02.945 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-22T06:29:03.071 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-22T06:29:03.109 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-22T06:29:03.137 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-22T06:29:03.171 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-22T06:29:03.215 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-22T06:29:03.253 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-22T06:29:03.292 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-22T06:29:03.322 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-22T06:29:03.356 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-22T06:29:03.390 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-22T06:29:03.431 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-22T06:29:03.466 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-22T06:29:03.555 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-22T06:29:03.605 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-22T06:29:03.636 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-22T06:29:03.684 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-22T06:29:03.792 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-22T06:29:03.846 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-22T06:29:04.597 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-22T06:29:04.652 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-22T06:29:04.685 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-22T06:29:04.789 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-22T06:29:04.844 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-22T06:29:04.933 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-22T06:29:04.970 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-22T06:29:05.147 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-22T06:29:05.176 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-22T06:29:05.253 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-22T06:29:05.917 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-22T06:29:06.064 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-22T06:29:07.329 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-22T06:29:07.434 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-22T06:29:07.462 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-22T06:29:07.495 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-22T06:29:07.806 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-22T06:29:07.841 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : librbd1-2:19.0.0-3156.g86f7587a.el9.x86_64 97/152 2024-04-22T06:29:07.923 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librbd1-2:19.0.0-3156.g86f7587a.el9.x86_64 97/152 2024-04-22T06:29:07.954 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rbd-2:19.0.0-3156.g86f7587a.el9.x86_64 98/152 2024-04-22T06:29:07.990 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-22T06:29:08.468 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-22T06:29:08.528 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librgw2-2:19.0.0-3156.g86f7587a.el9.x86_64 101/152 2024-04-22T06:29:08.542 INFO:teuthology.orchestra.run.smithi044.stdout:(150/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 748 kB/s | 16 MB 00:22 2024-04-22T06:29:08.544 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-04-22T06:29:08.545 INFO:teuthology.orchestra.run.smithi044.stdout:Total 5.3 MB/s | 218 MB 00:41 2024-04-22T06:29:08.845 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librgw2-2:19.0.0-3156.g86f7587a.el9.x86_64 101/152 2024-04-22T06:29:08.870 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rgw-2:19.0.0-3156.g86f7587a.el9.x86_64 102/152 2024-04-22T06:29:09.583 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-04-22T06:29:09.720 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-04-22T06:29:09.720 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-04-22T06:29:11.297 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 103/152 2024-04-22T06:29:11.333 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 103/152 2024-04-22T06:29:11.535 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-04-22T06:29:11.536 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-04-22T06:29:11.870 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 103/152 2024-04-22T06:29:11.903 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-base-2:19.0.0-3156.g86f7587a.el9.x86_64 104/152 2024-04-22T06:29:11.986 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.g86f7587a.el9.x86_64 104/152 2024-04-22T06:29:11.986 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-22T06:29:11.986 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-22T06:29:11.986 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:29:12.013 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-selinux-2:19.0.0-3156.g86f7587a.el9.x86_64 105/152 2024-04-22T06:29:17.453 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-04-22T06:29:17.529 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-22T06:29:17.562 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-22T06:29:17.606 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-22T06:29:17.953 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-22T06:29:18.088 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : librados2-2:19.0.0-3156.g86f7587a.el9.x86_64 5/152 2024-04-22T06:29:18.221 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librados2-2:19.0.0-3156.g86f7587a.el9.x86_64 5/152 2024-04-22T06:29:18.247 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephfs2-2:19.0.0-3156.g86f7587a.el9.x86_64 6/152 2024-04-22T06:29:18.332 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.g86f7587a.el9.x86_64 6/152 2024-04-22T06:29:18.373 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rados-2:19.0.0-3156.g86f7587a.el9.x86_64 7/152 2024-04-22T06:29:18.410 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-22T06:29:18.436 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-22T06:29:18.604 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-22T06:29:18.650 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-22T06:29:18.678 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephsqlite-2:19.0.0-3156.g86f7587a.el9.x86_64 12/152 2024-04-22T06:29:18.783 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.g86f7587a.el9.x86_64 12/152 2024-04-22T06:29:18.812 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libradosstriper1-2:19.0.0-3156.g86f7587a.el9.x86 13/152 2024-04-22T06:29:18.875 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.g86f7587a.el9.x86 13/152 2024-04-22T06:29:18.896 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-22T06:29:18.917 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-22T06:29:19.122 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-22T06:29:19.155 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-22T06:29:19.265 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-22T06:29:19.389 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-22T06:29:19.429 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-22T06:29:19.464 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-22T06:29:19.542 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-22T06:29:19.580 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-22T06:29:19.624 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-22T06:29:19.656 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-22T06:29:19.684 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-22T06:29:19.757 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-22T06:29:19.817 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-22T06:29:19.844 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-22T06:29:19.880 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-22T06:29:19.960 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-22T06:29:19.999 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ceph-common-2:19.0.0-3156.g86f7587a.el9. 30/152 2024-04-22T06:29:20.052 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.g86f7587a.el 31/152 2024-04-22T06:29:20.079 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cephfs-2:19.0.0-3156.g86f7587a.el9.x86_6 32/152 2024-04-22T06:29:20.156 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-22T06:29:20.205 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-22T06:29:20.283 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-22T06:29:20.440 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-22T06:29:20.499 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-22T06:29:21.073 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-22T06:29:21.163 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-22T06:29:21.204 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-22T06:29:21.248 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-22T06:29:21.286 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librados-devel-2:19.0.0-3156.g86f7587a.el9.x86_6 42/152 2024-04-22T06:29:21.319 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-22T06:29:21.377 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-22T06:29:21.451 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-22T06:29:21.486 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-22T06:29:21.523 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-22T06:29:21.571 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-22T06:29:21.605 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-22T06:29:21.650 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-22T06:29:21.691 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-22T06:29:21.838 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-22T06:29:21.905 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-22T06:29:21.935 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-22T06:29:21.977 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-22T06:29:22.076 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-22T06:29:22.117 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-22T06:29:22.158 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-22T06:29:22.191 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-22T06:29:22.216 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-22T06:29:22.295 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-22T06:29:22.434 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-22T06:29:22.474 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-22T06:29:22.510 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-22T06:29:22.544 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-22T06:29:22.589 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-22T06:29:22.635 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-22T06:29:22.674 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-22T06:29:22.704 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-22T06:29:22.738 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-22T06:29:22.772 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-22T06:29:22.823 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-22T06:29:22.864 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-22T06:29:22.957 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-22T06:29:23.004 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-22T06:29:23.034 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-22T06:29:23.083 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-22T06:29:23.196 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-22T06:29:23.254 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-22T06:29:23.413 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.g86f7587a.el9.x86_64 105/152 2024-04-22T06:29:23.413 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /sys 2024-04-22T06:29:23.413 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /proc 2024-04-22T06:29:23.413 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /mnt 2024-04-22T06:29:23.413 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /var/tmp 2024-04-22T06:29:23.414 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /home 2024-04-22T06:29:23.414 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /root 2024-04-22T06:29:23.414 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /tmp 2024-04-22T06:29:23.414 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:29:23.615 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-volume-2:19.0.0-3156.g86f7587a.el9.noarch 106/152 2024-04-22T06:29:23.630 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.g86f7587a.el9.noarch 106/152 2024-04-22T06:29:23.631 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T06:29:23.631 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-22T06:29:23.631 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:29:24.050 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-22T06:29:24.091 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-22T06:29:24.125 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-22T06:29:24.232 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-22T06:29:24.285 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-22T06:29:24.377 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-22T06:29:24.419 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-22T06:29:24.601 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-22T06:29:24.633 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-22T06:29:24.714 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-22T06:29:25.101 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-osd-2:19.0.0-3156.g86f7587a.el9.x86_64 107/152 2024-04-22T06:29:25.126 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.g86f7587a.el9.x86_64 107/152 2024-04-22T06:29:25.127 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T06:29:25.127 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-22T06:29:25.127 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T06:29:25.127 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T06:29:25.127 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:29:25.361 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mds-2:19.0.0-3156.g86f7587a.el9.x86_64 108/152 2024-04-22T06:29:25.383 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.g86f7587a.el9.x86_64 108/152 2024-04-22T06:29:25.383 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T06:29:25.383 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-22T06:29:25.383 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T06:29:25.383 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T06:29:25.383 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:29:25.414 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-22T06:29:25.529 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-22T06:29:25.828 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mon-2:19.0.0-3156.g86f7587a.el9.x86_64 109/152 2024-04-22T06:29:25.850 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.g86f7587a.el9.x86_64 109/152 2024-04-22T06:29:25.850 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T06:29:25.850 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-22T06:29:25.850 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T06:29:25.851 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T06:29:25.851 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:29:25.950 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-22T06:29:26.007 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-22T06:29:26.070 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-22T06:29:26.264 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-22T06:29:26.312 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-22T06:29:26.398 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-22T06:29:26.494 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-22T06:29:26.632 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-22T06:29:26.670 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-22T06:29:26.711 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-22T06:29:26.747 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-22T06:29:26.868 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-22T06:29:26.953 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-22T06:29:26.982 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-22T06:29:27.023 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-22T06:29:27.190 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 121/152 2024-04-22T06:29:27.220 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 121/152 2024-04-22T06:29:27.252 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.g86f7587a.e 122/152 2024-04-22T06:29:27.313 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.g86f7587a. 123/152 2024-04-22T06:29:27.351 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-22T06:29:27.351 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.g86f7587a.el9.noa 124/152 2024-04-22T06:29:27.418 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : librbd1-2:19.0.0-3156.g86f7587a.el9.x86_64 97/152 2024-04-22T06:29:27.518 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librbd1-2:19.0.0-3156.g86f7587a.el9.x86_64 97/152 2024-04-22T06:29:27.549 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rbd-2:19.0.0-3156.g86f7587a.el9.x86_64 98/152 2024-04-22T06:29:27.585 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-22T06:29:28.090 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-22T06:29:28.122 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librgw2-2:19.0.0-3156.g86f7587a.el9.x86_64 101/152 2024-04-22T06:29:28.193 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librgw2-2:19.0.0-3156.g86f7587a.el9.x86_64 101/152 2024-04-22T06:29:28.223 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rgw-2:19.0.0-3156.g86f7587a.el9.x86_64 102/152 2024-04-22T06:29:28.388 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.g86f7587a.el9.noa 124/152 2024-04-22T06:29:28.428 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.g86f7587a.el9.n 125/152 2024-04-22T06:29:29.387 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.g86f7587a.el9.n 125/152 2024-04-22T06:29:29.412 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.g86f 126/152 2024-04-22T06:29:29.512 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.g86f 126/152 2024-04-22T06:29:29.674 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.g86f7587a.el 127/152 2024-04-22T06:29:29.706 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-2:19.0.0-3156.g86f7587a.el9.x86_64 128/152 2024-04-22T06:29:29.728 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.g86f7587a.el9.x86_64 128/152 2024-04-22T06:29:29.728 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T06:29:29.728 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-22T06:29:29.728 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T06:29:29.729 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T06:29:29.729 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:29:29.773 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.g86f7587a.el9.noarch 129/152 2024-04-22T06:29:29.784 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.g86f7587a.el9.noarch 129/152 2024-04-22T06:29:29.882 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-2:19.0.0-3156.g86f7587a.el9.x86_64 130/152 2024-04-22T06:29:29.930 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 131/152 2024-04-22T06:29:30.566 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 103/152 2024-04-22T06:29:30.603 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 103/152 2024-04-22T06:29:30.866 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-22T06:29:30.900 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-radosgw-2:19.0.0-3156.g86f7587a.el9.x86_64 133/152 2024-04-22T06:29:30.923 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.g86f7587a.el9.x86_64 133/152 2024-04-22T06:29:30.923 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T06:29:30.924 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-22T06:29:30.924 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T06:29:30.924 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T06:29:30.924 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:29:30.968 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.g86f75 134/152 2024-04-22T06:29:30.990 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.g86f75 134/152 2024-04-22T06:29:30.990 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T06:29:30.990 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-22T06:29:30.990 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:29:31.035 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : cephfs-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 135/152 2024-04-22T06:29:31.058 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 135/152 2024-04-22T06:29:31.058 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T06:29:31.058 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-22T06:29:31.058 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-22T06:29:31.058 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-22T06:29:31.058 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:29:31.171 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 103/152 2024-04-22T06:29:31.215 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-base-2:19.0.0-3156.g86f7587a.el9.x86_64 104/152 2024-04-22T06:29:31.289 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.g86f7587a.el9.x86_64 104/152 2024-04-22T06:29:31.289 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-22T06:29:31.289 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-22T06:29:31.290 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:29:31.316 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-selinux-2:19.0.0-3156.g86f7587a.el9.x86_64 105/152 2024-04-22T06:29:31.361 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 136/152 2024-04-22T06:29:31.383 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 136/152 2024-04-22T06:29:31.384 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T06:29:31.384 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-22T06:29:31.384 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T06:29:31.384 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T06:29:31.384 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:29:35.955 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-test-2:19.0.0-3156.g86f7587a.el9.x86_64 137/152 2024-04-22T06:29:36.005 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 138/152 2024-04-22T06:29:36.054 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-nbd-2:19.0.0-3156.g86f7587a.el9.x86_64 139/152 2024-04-22T06:29:36.086 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-22T06:29:36.114 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-22T06:29:36.296 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : cephfs-shell-2:19.0.0-3156.g86f7587a.el9.x86_64 142/152 2024-04-22T06:29:36.343 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-22T06:29:36.373 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephfs-devel-2:19.0.0-3156.g86f7587a.el9.x86_ 144/152 2024-04-22T06:29:36.985 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T06:29:37.029 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T06:29:37.049 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T06:29:37.828 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : cephfs-top-2:19.0.0-3156.g86f7587a.el9.noarch 146/152 2024-04-22T06:29:37.916 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-22T06:29:37.958 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-22T06:29:38.041 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-22T06:29:38.071 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-22T06:29:38.071 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-22T06:29:38.154 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-22T06:29:38.155 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T06:29:43.018 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.g86f7587a.el9.x86_64 105/152 2024-04-22T06:29:43.019 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /sys 2024-04-22T06:29:43.019 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /proc 2024-04-22T06:29:43.019 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /mnt 2024-04-22T06:29:43.019 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /var/tmp 2024-04-22T06:29:43.019 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /home 2024-04-22T06:29:43.019 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /root 2024-04-22T06:29:43.019 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /tmp 2024-04-22T06:29:43.019 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:29:43.202 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-volume-2:19.0.0-3156.g86f7587a.el9.noarch 106/152 2024-04-22T06:29:43.218 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.g86f7587a.el9.noarch 106/152 2024-04-22T06:29:43.218 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T06:29:43.218 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-22T06:29:43.218 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:29:44.182 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T06:29:44.182 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-2:19.0.0-3156.g86f7587a.el9.x86_64 1/152 2024-04-22T06:29:44.182 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-base-2:19.0.0-3156.g86f7587a.el9.x86_64 2/152 2024-04-22T06:29:44.182 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 3/152 2024-04-22T06:29:44.182 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 4/152 2024-04-22T06:29:44.182 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.g86f75 5/152 2024-04-22T06:29:44.183 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mds-2:19.0.0-3156.g86f7587a.el9.x86_64 6/152 2024-04-22T06:29:44.183 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-2:19.0.0-3156.g86f7587a.el9.x86_64 7/152 2024-04-22T06:29:44.183 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mon-2:19.0.0-3156.g86f7587a.el9.x86_64 8/152 2024-04-22T06:29:44.183 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-osd-2:19.0.0-3156.g86f7587a.el9.x86_64 9/152 2024-04-22T06:29:44.183 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.g86f7587a.el9.x86_64 10/152 2024-04-22T06:29:44.183 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-selinux-2:19.0.0-3156.g86f7587a.el9.x86_64 11/152 2024-04-22T06:29:44.183 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-test-2:19.0.0-3156.g86f7587a.el9.x86_64 12/152 2024-04-22T06:29:44.183 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 13/152 2024-04-22T06:29:44.183 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : cephfs-shell-2:19.0.0-3156.g86f7587a.el9.x86_64 14/152 2024-04-22T06:29:44.183 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.g86f7587a.el9.x86_ 15/152 2024-04-22T06:29:44.183 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephfs2-2:19.0.0-3156.g86f7587a.el9.x86_64 16/152 2024-04-22T06:29:44.183 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephsqlite-2:19.0.0-3156.g86f7587a.el9.x86_64 17/152 2024-04-22T06:29:44.183 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados-devel-2:19.0.0-3156.g86f7587a.el9.x86_6 18/152 2024-04-22T06:29:44.183 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libradosstriper1-2:19.0.0-3156.g86f7587a.el9.x86 19/152 2024-04-22T06:29:44.184 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librgw2-2:19.0.0-3156.g86f7587a.el9.x86_64 20/152 2024-04-22T06:29:44.184 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.g86f7587a.el 21/152 2024-04-22T06:29:44.184 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.g86f7587a.el9. 22/152 2024-04-22T06:29:44.184 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cephfs-2:19.0.0-3156.g86f7587a.el9.x86_6 23/152 2024-04-22T06:29:44.184 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rados-2:19.0.0-3156.g86f7587a.el9.x86_64 24/152 2024-04-22T06:29:44.184 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rbd-2:19.0.0-3156.g86f7587a.el9.x86_64 25/152 2024-04-22T06:29:44.184 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rgw-2:19.0.0-3156.g86f7587a.el9.x86_64 26/152 2024-04-22T06:29:44.184 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 27/152 2024-04-22T06:29:44.184 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 28/152 2024-04-22T06:29:44.184 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-nbd-2:19.0.0-3156.g86f7587a.el9.x86_64 29/152 2024-04-22T06:29:44.184 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.g86f7587a. 30/152 2024-04-22T06:29:44.184 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.g86f7587a.el9.noa 31/152 2024-04-22T06:29:44.184 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.g86f7587a.el9.n 32/152 2024-04-22T06:29:44.184 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.g86f 33/152 2024-04-22T06:29:44.184 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.g86f7587a.el 34/152 2024-04-22T06:29:44.185 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.g86f7587a.el9.noarch 35/152 2024-04-22T06:29:44.185 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.g86f7587a.e 36/152 2024-04-22T06:29:44.185 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-volume-2:19.0.0-3156.g86f7587a.el9.noarch 37/152 2024-04-22T06:29:44.185 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 38/152 2024-04-22T06:29:44.185 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : cephfs-top-2:19.0.0-3156.g86f7587a.el9.noarch 39/152 2024-04-22T06:29:44.185 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-22T06:29:44.185 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-22T06:29:44.185 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-22T06:29:44.185 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-22T06:29:44.185 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-22T06:29:44.185 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-22T06:29:44.185 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-22T06:29:44.185 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-22T06:29:44.185 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-22T06:29:44.186 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-22T06:29:44.186 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-22T06:29:44.186 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-22T06:29:44.186 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-22T06:29:44.186 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-22T06:29:44.186 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-22T06:29:44.186 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-22T06:29:44.186 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-22T06:29:44.186 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-22T06:29:44.186 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-22T06:29:44.186 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-22T06:29:44.186 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-22T06:29:44.186 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-22T06:29:44.187 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-22T06:29:44.187 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-22T06:29:44.187 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-22T06:29:44.187 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-22T06:29:44.187 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-22T06:29:44.187 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-22T06:29:44.187 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-22T06:29:44.187 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-22T06:29:44.187 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-22T06:29:44.187 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-22T06:29:44.187 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-22T06:29:44.187 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-22T06:29:44.188 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-22T06:29:44.188 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-22T06:29:44.188 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-22T06:29:44.188 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-22T06:29:44.188 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-22T06:29:44.188 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-22T06:29:44.188 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-22T06:29:44.188 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-22T06:29:44.188 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-22T06:29:44.188 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-22T06:29:44.188 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-22T06:29:44.188 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-22T06:29:44.188 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-22T06:29:44.188 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-22T06:29:44.188 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-22T06:29:44.189 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-22T06:29:44.189 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-22T06:29:44.189 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-22T06:29:44.189 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-22T06:29:44.189 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-22T06:29:44.189 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-22T06:29:44.189 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-22T06:29:44.189 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-22T06:29:44.189 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-22T06:29:44.189 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-22T06:29:44.189 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-22T06:29:44.189 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-22T06:29:44.189 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-22T06:29:44.189 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-22T06:29:44.190 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-22T06:29:44.190 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-22T06:29:44.190 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-22T06:29:44.190 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-22T06:29:44.190 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-22T06:29:44.190 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-22T06:29:44.190 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-22T06:29:44.190 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-22T06:29:44.190 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-22T06:29:44.190 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-22T06:29:44.190 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-22T06:29:44.190 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-22T06:29:44.190 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-22T06:29:44.191 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-22T06:29:44.191 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-22T06:29:44.191 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-22T06:29:44.191 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-22T06:29:44.191 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-22T06:29:44.191 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-22T06:29:44.191 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-22T06:29:44.191 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-22T06:29:44.191 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-22T06:29:44.191 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-22T06:29:44.191 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-22T06:29:44.191 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-22T06:29:44.191 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-22T06:29:44.191 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-22T06:29:44.191 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-22T06:29:44.192 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-22T06:29:44.192 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-22T06:29:44.192 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-22T06:29:44.192 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-22T06:29:44.192 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-22T06:29:44.192 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-22T06:29:44.192 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-22T06:29:44.192 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-22T06:29:44.192 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-22T06:29:44.192 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-22T06:29:44.192 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-22T06:29:44.192 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-22T06:29:44.192 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-22T06:29:44.192 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-22T06:29:44.193 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-22T06:29:44.193 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-22T06:29:44.193 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-22T06:29:44.193 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-22T06:29:44.193 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados2-2:19.0.0-3156.g86f7587a.el9.x86_64 149/152 2024-04-22T06:29:44.193 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-22T06:29:44.193 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librbd1-2:19.0.0-3156.g86f7587a.el9.x86_64 151/152 2024-04-22T06:29:44.748 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-osd-2:19.0.0-3156.g86f7587a.el9.x86_64 107/152 2024-04-22T06:29:44.775 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.g86f7587a.el9.x86_64 107/152 2024-04-22T06:29:44.775 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T06:29:44.775 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-22T06:29:44.775 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T06:29:44.775 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T06:29:44.776 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:29:45.007 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mds-2:19.0.0-3156.g86f7587a.el9.x86_64 108/152 2024-04-22T06:29:45.031 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.g86f7587a.el9.x86_64 108/152 2024-04-22T06:29:45.031 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T06:29:45.031 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-22T06:29:45.031 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T06:29:45.031 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T06:29:45.031 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:29:45.491 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mon-2:19.0.0-3156.g86f7587a.el9.x86_64 109/152 2024-04-22T06:29:45.515 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.g86f7587a.el9.x86_64 109/152 2024-04-22T06:29:45.515 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T06:29:45.515 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-22T06:29:45.515 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T06:29:45.515 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T06:29:45.515 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:29:45.623 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-22T06:29:45.681 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-22T06:29:45.744 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-22T06:29:45.943 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-22T06:29:45.993 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-22T06:29:46.074 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-22T06:29:46.172 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-22T06:29:46.316 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-22T06:29:46.356 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T06:29:46.356 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:29:46.357 INFO:teuthology.orchestra.run.smithi070.stdout:Upgraded: 2024-04-22T06:29:46.357 INFO:teuthology.orchestra.run.smithi070.stdout: librados2-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:29:46.357 INFO:teuthology.orchestra.run.smithi070.stdout: librbd1-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:29:46.357 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-04-22T06:29:46.357 INFO:teuthology.orchestra.run.smithi070.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-22T06:29:46.357 INFO:teuthology.orchestra.run.smithi070.stdout: automake-1.16.2-8.el9.noarch 2024-04-22T06:29:46.357 INFO:teuthology.orchestra.run.smithi070.stdout: bison-3.7.4-5.el9.x86_64 2024-04-22T06:29:46.357 INFO:teuthology.orchestra.run.smithi070.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-22T06:29:46.357 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:29:46.357 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-base-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:29:46.357 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:29:46.357 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:29:46.357 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-grafana-dashboards-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T06:29:46.358 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-immutable-object-cache-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:29:46.358 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mds-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:29:46.358 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:29:46.358 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T06:29:46.358 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-dashboard-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T06:29:46.358 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T06:29:46.358 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-modules-core-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T06:29:46.358 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-rook-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T06:29:46.358 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mon-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:29:46.358 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-osd-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:29:46.358 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-prometheus-alerts-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T06:29:46.358 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-radosgw-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:29:46.359 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-selinux-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:29:46.359 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-test-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:29:46.359 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-volume-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T06:29:46.359 INFO:teuthology.orchestra.run.smithi070.stdout: cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T06:29:46.359 INFO:teuthology.orchestra.run.smithi070.stdout: cephfs-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:29:46.359 INFO:teuthology.orchestra.run.smithi070.stdout: cephfs-shell-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:29:46.359 INFO:teuthology.orchestra.run.smithi070.stdout: cephfs-top-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T06:29:46.359 INFO:teuthology.orchestra.run.smithi070.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-22T06:29:46.359 INFO:teuthology.orchestra.run.smithi070.stdout: flex-2.6.4-9.el9.x86_64 2024-04-22T06:29:46.359 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-22T06:29:46.359 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-22T06:29:46.360 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-22T06:29:46.360 INFO:teuthology.orchestra.run.smithi070.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-22T06:29:46.360 INFO:teuthology.orchestra.run.smithi070.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-22T06:29:46.360 INFO:teuthology.orchestra.run.smithi070.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-22T06:29:46.360 INFO:teuthology.orchestra.run.smithi070.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-22T06:29:46.360 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-22T06:29:46.360 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-22T06:29:46.360 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs-devel-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:29:46.360 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs2-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:29:46.360 INFO:teuthology.orchestra.run.smithi070.stdout: libcephsqlite-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:29:46.360 INFO:teuthology.orchestra.run.smithi070.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-22T06:29:46.360 INFO:teuthology.orchestra.run.smithi070.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-22T06:29:46.360 INFO:teuthology.orchestra.run.smithi070.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-22T06:29:46.361 INFO:teuthology.orchestra.run.smithi070.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-22T06:29:46.361 INFO:teuthology.orchestra.run.smithi070.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-22T06:29:46.361 INFO:teuthology.orchestra.run.smithi070.stdout: librados-devel-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:29:46.361 INFO:teuthology.orchestra.run.smithi070.stdout: libradosstriper1-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:29:46.361 INFO:teuthology.orchestra.run.smithi070.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-22T06:29:46.361 INFO:teuthology.orchestra.run.smithi070.stdout: librgw2-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:29:46.361 INFO:teuthology.orchestra.run.smithi070.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-22T06:29:46.361 INFO:teuthology.orchestra.run.smithi070.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-22T06:29:46.361 INFO:teuthology.orchestra.run.smithi070.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-22T06:29:46.361 INFO:teuthology.orchestra.run.smithi070.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-22T06:29:46.361 INFO:teuthology.orchestra.run.smithi070.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-22T06:29:46.361 INFO:teuthology.orchestra.run.smithi070.stdout: lua-5.4.4-4.el9.x86_64 2024-04-22T06:29:46.361 INFO:teuthology.orchestra.run.smithi070.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-22T06:29:46.361 INFO:teuthology.orchestra.run.smithi070.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-22T06:29:46.361 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-22T06:29:46.362 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-22T06:29:46.362 INFO:teuthology.orchestra.run.smithi070.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-22T06:29:46.362 INFO:teuthology.orchestra.run.smithi070.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-22T06:29:46.362 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-13.14-1.el9.x86_64 2024-04-22T06:29:46.362 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-22T06:29:46.362 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-22T06:29:46.362 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-22T06:29:46.362 INFO:teuthology.orchestra.run.smithi070.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-22T06:29:46.362 INFO:teuthology.orchestra.run.smithi070.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-22T06:29:46.362 INFO:teuthology.orchestra.run.smithi070.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-22T06:29:46.362 INFO:teuthology.orchestra.run.smithi070.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-22T06:29:46.362 INFO:teuthology.orchestra.run.smithi070.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-22T06:29:46.362 INFO:teuthology.orchestra.run.smithi070.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-22T06:29:46.362 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-22T06:29:46.363 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-argparse-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:29:46.363 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:29:46.363 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cephfs-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:29:46.363 INFO:teuthology.orchestra.run.smithi070.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-22T06:29:46.363 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-22T06:29:46.363 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-22T06:29:46.363 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-22T06:29:46.363 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-22T06:29:46.363 INFO:teuthology.orchestra.run.smithi070.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-22T06:29:46.363 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-22T06:29:46.363 INFO:teuthology.orchestra.run.smithi070.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-22T06:29:46.363 INFO:teuthology.orchestra.run.smithi070.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-22T06:29:46.363 INFO:teuthology.orchestra.run.smithi070.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-22T06:29:46.363 INFO:teuthology.orchestra.run.smithi070.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-22T06:29:46.364 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-22T06:29:46.364 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-22T06:29:46.364 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-22T06:29:46.364 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-22T06:29:46.364 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-22T06:29:46.364 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-22T06:29:46.364 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-22T06:29:46.364 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-22T06:29:46.364 INFO:teuthology.orchestra.run.smithi070.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-22T06:29:46.364 INFO:teuthology.orchestra.run.smithi070.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-22T06:29:46.364 INFO:teuthology.orchestra.run.smithi070.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-22T06:29:46.364 INFO:teuthology.orchestra.run.smithi070.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-22T06:29:46.364 INFO:teuthology.orchestra.run.smithi070.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-22T06:29:46.364 INFO:teuthology.orchestra.run.smithi070.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-22T06:29:46.365 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-22T06:29:46.365 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-22T06:29:46.365 INFO:teuthology.orchestra.run.smithi070.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-22T06:29:46.365 INFO:teuthology.orchestra.run.smithi070.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-22T06:29:46.365 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-22T06:29:46.365 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ply-3.11-14.el9.noarch 2024-04-22T06:29:46.365 INFO:teuthology.orchestra.run.smithi070.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-22T06:29:46.365 INFO:teuthology.orchestra.run.smithi070.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-22T06:29:46.365 INFO:teuthology.orchestra.run.smithi070.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-22T06:29:46.365 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-22T06:29:46.365 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-22T06:29:46.365 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-22T06:29:46.365 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-22T06:29:46.365 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-22T06:29:46.366 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-22T06:29:46.366 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rados-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:29:46.366 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rbd-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:29:46.366 INFO:teuthology.orchestra.run.smithi070.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-22T06:29:46.366 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-22T06:29:46.366 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-22T06:29:46.366 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rgw-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:29:46.366 INFO:teuthology.orchestra.run.smithi070.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-22T06:29:46.366 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-22T06:29:46.366 INFO:teuthology.orchestra.run.smithi070.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-22T06:29:46.366 INFO:teuthology.orchestra.run.smithi070.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-22T06:29:46.366 INFO:teuthology.orchestra.run.smithi070.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-22T06:29:46.366 INFO:teuthology.orchestra.run.smithi070.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-22T06:29:46.366 INFO:teuthology.orchestra.run.smithi070.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-22T06:29:46.367 INFO:teuthology.orchestra.run.smithi070.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-22T06:29:46.367 INFO:teuthology.orchestra.run.smithi070.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-22T06:29:46.367 INFO:teuthology.orchestra.run.smithi070.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-22T06:29:46.367 INFO:teuthology.orchestra.run.smithi070.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-22T06:29:46.367 INFO:teuthology.orchestra.run.smithi070.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-22T06:29:46.367 INFO:teuthology.orchestra.run.smithi070.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-22T06:29:46.367 INFO:teuthology.orchestra.run.smithi070.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-22T06:29:46.367 INFO:teuthology.orchestra.run.smithi070.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-22T06:29:46.367 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:29:46.367 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:29:46.367 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-nbd-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:29:46.367 INFO:teuthology.orchestra.run.smithi070.stdout: re2-1:20211101-3.el9.x86_64 2024-04-22T06:29:46.367 INFO:teuthology.orchestra.run.smithi070.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-22T06:29:46.367 INFO:teuthology.orchestra.run.smithi070.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-22T06:29:46.367 INFO:teuthology.orchestra.run.smithi070.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-22T06:29:46.367 INFO:teuthology.orchestra.run.smithi070.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-22T06:29:46.367 INFO:teuthology.orchestra.run.smithi070.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-22T06:29:46.368 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-22T06:29:46.368 INFO:teuthology.orchestra.run.smithi070.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-22T06:29:46.368 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:29:46.368 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-04-22T06:29:46.391 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-22T06:29:46.418 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-22T06:29:46.724 DEBUG:teuthology.parallel:result is None 2024-04-22T06:29:46.797 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 121/152 2024-04-22T06:29:46.825 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 121/152 2024-04-22T06:29:46.858 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.g86f7587a.e 122/152 2024-04-22T06:29:46.922 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.g86f7587a. 123/152 2024-04-22T06:29:46.948 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.g86f7587a.el9.noa 124/152 2024-04-22T06:29:48.040 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.g86f7587a.el9.noa 124/152 2024-04-22T06:29:48.075 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.g86f7587a.el9.n 125/152 2024-04-22T06:29:49.086 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.g86f7587a.el9.n 125/152 2024-04-22T06:29:49.109 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.g86f 126/152 2024-04-22T06:29:49.218 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.g86f 126/152 2024-04-22T06:29:49.388 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.g86f7587a.el 127/152 2024-04-22T06:29:49.411 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-2:19.0.0-3156.g86f7587a.el9.x86_64 128/152 2024-04-22T06:29:49.436 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.g86f7587a.el9.x86_64 128/152 2024-04-22T06:29:49.436 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T06:29:49.436 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-22T06:29:49.436 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T06:29:49.436 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T06:29:49.436 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:29:49.487 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.g86f7587a.el9.noarch 129/152 2024-04-22T06:29:49.497 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.g86f7587a.el9.noarch 129/152 2024-04-22T06:29:49.599 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-2:19.0.0-3156.g86f7587a.el9.x86_64 130/152 2024-04-22T06:29:49.653 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 131/152 2024-04-22T06:29:50.625 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-22T06:29:50.656 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-radosgw-2:19.0.0-3156.g86f7587a.el9.x86_64 133/152 2024-04-22T06:29:50.679 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.g86f7587a.el9.x86_64 133/152 2024-04-22T06:29:50.680 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T06:29:50.680 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-22T06:29:50.680 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T06:29:50.680 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T06:29:50.680 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:29:50.726 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.g86f75 134/152 2024-04-22T06:29:50.750 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.g86f75 134/152 2024-04-22T06:29:50.750 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T06:29:50.750 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-22T06:29:50.750 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:29:50.799 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephfs-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 135/152 2024-04-22T06:29:50.823 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 135/152 2024-04-22T06:29:50.823 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T06:29:50.823 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-22T06:29:50.823 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-22T06:29:50.823 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-22T06:29:50.823 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:29:51.141 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 136/152 2024-04-22T06:29:51.165 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 136/152 2024-04-22T06:29:51.165 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T06:29:51.165 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-22T06:29:51.165 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T06:29:51.166 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T06:29:51.166 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:29:56.020 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-test-2:19.0.0-3156.g86f7587a.el9.x86_64 137/152 2024-04-22T06:29:56.061 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 138/152 2024-04-22T06:29:56.103 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-nbd-2:19.0.0-3156.g86f7587a.el9.x86_64 139/152 2024-04-22T06:29:56.134 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-22T06:29:56.161 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-22T06:29:56.364 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephfs-shell-2:19.0.0-3156.g86f7587a.el9.x86_64 142/152 2024-04-22T06:29:56.416 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-22T06:29:56.445 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephfs-devel-2:19.0.0-3156.g86f7587a.el9.x86_ 144/152 2024-04-22T06:29:57.066 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T06:29:57.109 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T06:29:57.130 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T06:29:57.973 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephfs-top-2:19.0.0-3156.g86f7587a.el9.noarch 146/152 2024-04-22T06:29:58.055 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-22T06:29:58.090 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-22T06:29:58.184 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-22T06:29:58.218 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-22T06:29:58.218 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-22T06:29:58.301 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-22T06:29:58.302 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T06:30:04.170 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T06:30:04.171 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-2:19.0.0-3156.g86f7587a.el9.x86_64 1/152 2024-04-22T06:30:04.171 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-base-2:19.0.0-3156.g86f7587a.el9.x86_64 2/152 2024-04-22T06:30:04.171 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 3/152 2024-04-22T06:30:04.171 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 4/152 2024-04-22T06:30:04.171 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.g86f75 5/152 2024-04-22T06:30:04.171 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mds-2:19.0.0-3156.g86f7587a.el9.x86_64 6/152 2024-04-22T06:30:04.171 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-2:19.0.0-3156.g86f7587a.el9.x86_64 7/152 2024-04-22T06:30:04.171 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mon-2:19.0.0-3156.g86f7587a.el9.x86_64 8/152 2024-04-22T06:30:04.172 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-osd-2:19.0.0-3156.g86f7587a.el9.x86_64 9/152 2024-04-22T06:30:04.172 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.g86f7587a.el9.x86_64 10/152 2024-04-22T06:30:04.172 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-selinux-2:19.0.0-3156.g86f7587a.el9.x86_64 11/152 2024-04-22T06:30:04.172 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-test-2:19.0.0-3156.g86f7587a.el9.x86_64 12/152 2024-04-22T06:30:04.172 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 13/152 2024-04-22T06:30:04.172 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephfs-shell-2:19.0.0-3156.g86f7587a.el9.x86_64 14/152 2024-04-22T06:30:04.172 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.g86f7587a.el9.x86_ 15/152 2024-04-22T06:30:04.172 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephfs2-2:19.0.0-3156.g86f7587a.el9.x86_64 16/152 2024-04-22T06:30:04.172 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephsqlite-2:19.0.0-3156.g86f7587a.el9.x86_64 17/152 2024-04-22T06:30:04.173 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados-devel-2:19.0.0-3156.g86f7587a.el9.x86_6 18/152 2024-04-22T06:30:04.173 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libradosstriper1-2:19.0.0-3156.g86f7587a.el9.x86 19/152 2024-04-22T06:30:04.173 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librgw2-2:19.0.0-3156.g86f7587a.el9.x86_64 20/152 2024-04-22T06:30:04.173 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.g86f7587a.el 21/152 2024-04-22T06:30:04.173 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.g86f7587a.el9. 22/152 2024-04-22T06:30:04.173 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cephfs-2:19.0.0-3156.g86f7587a.el9.x86_6 23/152 2024-04-22T06:30:04.173 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rados-2:19.0.0-3156.g86f7587a.el9.x86_64 24/152 2024-04-22T06:30:04.173 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rbd-2:19.0.0-3156.g86f7587a.el9.x86_64 25/152 2024-04-22T06:30:04.173 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rgw-2:19.0.0-3156.g86f7587a.el9.x86_64 26/152 2024-04-22T06:30:04.173 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 27/152 2024-04-22T06:30:04.173 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 28/152 2024-04-22T06:30:04.174 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-nbd-2:19.0.0-3156.g86f7587a.el9.x86_64 29/152 2024-04-22T06:30:04.174 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.g86f7587a. 30/152 2024-04-22T06:30:04.174 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.g86f7587a.el9.noa 31/152 2024-04-22T06:30:04.174 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.g86f7587a.el9.n 32/152 2024-04-22T06:30:04.174 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.g86f 33/152 2024-04-22T06:30:04.174 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.g86f7587a.el 34/152 2024-04-22T06:30:04.174 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.g86f7587a.el9.noarch 35/152 2024-04-22T06:30:04.174 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.g86f7587a.e 36/152 2024-04-22T06:30:04.174 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-volume-2:19.0.0-3156.g86f7587a.el9.noarch 37/152 2024-04-22T06:30:04.174 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 38/152 2024-04-22T06:30:04.174 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephfs-top-2:19.0.0-3156.g86f7587a.el9.noarch 39/152 2024-04-22T06:30:04.174 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-22T06:30:04.175 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-22T06:30:04.175 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-22T06:30:04.175 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-22T06:30:04.175 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-22T06:30:04.175 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-22T06:30:04.175 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-22T06:30:04.175 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-22T06:30:04.175 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-22T06:30:04.175 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-22T06:30:04.175 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-22T06:30:04.175 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-22T06:30:04.175 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-22T06:30:04.175 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-22T06:30:04.175 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-22T06:30:04.176 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-22T06:30:04.176 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-22T06:30:04.176 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-22T06:30:04.176 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-22T06:30:04.176 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-22T06:30:04.176 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-22T06:30:04.176 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-22T06:30:04.176 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-22T06:30:04.176 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-22T06:30:04.176 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-22T06:30:04.176 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-22T06:30:04.176 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-22T06:30:04.176 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-22T06:30:04.176 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-22T06:30:04.177 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-22T06:30:04.177 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-22T06:30:04.177 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-22T06:30:04.177 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-22T06:30:04.177 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-22T06:30:04.177 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-22T06:30:04.177 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-22T06:30:04.177 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-22T06:30:04.177 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-22T06:30:04.177 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-22T06:30:04.177 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-22T06:30:04.177 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-22T06:30:04.177 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-22T06:30:04.177 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-22T06:30:04.178 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-22T06:30:04.178 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-22T06:30:04.178 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-22T06:30:04.178 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-22T06:30:04.178 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-22T06:30:04.178 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-22T06:30:04.178 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-22T06:30:04.178 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-22T06:30:04.178 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-22T06:30:04.178 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-22T06:30:04.178 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-22T06:30:04.178 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-22T06:30:04.178 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-22T06:30:04.179 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-22T06:30:04.179 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-22T06:30:04.179 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-22T06:30:04.179 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-22T06:30:04.179 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-22T06:30:04.179 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-22T06:30:04.179 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-22T06:30:04.179 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-22T06:30:04.179 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-22T06:30:04.179 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-22T06:30:04.179 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-22T06:30:04.179 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-22T06:30:04.179 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-22T06:30:04.179 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-22T06:30:04.180 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-22T06:30:04.180 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-22T06:30:04.180 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-22T06:30:04.180 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-22T06:30:04.180 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-22T06:30:04.180 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-22T06:30:04.180 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-22T06:30:04.180 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-22T06:30:04.180 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-22T06:30:04.180 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-22T06:30:04.180 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-22T06:30:04.180 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-22T06:30:04.180 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-22T06:30:04.180 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-22T06:30:04.181 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-22T06:30:04.181 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-22T06:30:04.181 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-22T06:30:04.181 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-22T06:30:04.181 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-22T06:30:04.181 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-22T06:30:04.181 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-22T06:30:04.181 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-22T06:30:04.181 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-22T06:30:04.181 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-22T06:30:04.181 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-22T06:30:04.181 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-22T06:30:04.181 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-22T06:30:04.181 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-22T06:30:04.181 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-22T06:30:04.182 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-22T06:30:04.182 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-22T06:30:04.182 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-22T06:30:04.182 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-22T06:30:04.182 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-22T06:30:04.182 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-22T06:30:04.182 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-22T06:30:04.182 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-22T06:30:04.182 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-22T06:30:04.182 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-22T06:30:04.182 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados2-2:19.0.0-3156.g86f7587a.el9.x86_64 149/152 2024-04-22T06:30:04.182 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-22T06:30:04.182 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librbd1-2:19.0.0-3156.g86f7587a.el9.x86_64 151/152 2024-04-22T06:30:06.198 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T06:30:06.198 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:30:06.198 INFO:teuthology.orchestra.run.smithi044.stdout:Upgraded: 2024-04-22T06:30:06.198 INFO:teuthology.orchestra.run.smithi044.stdout: librados2-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:30:06.198 INFO:teuthology.orchestra.run.smithi044.stdout: librbd1-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:30:06.198 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-04-22T06:30:06.198 INFO:teuthology.orchestra.run.smithi044.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-22T06:30:06.198 INFO:teuthology.orchestra.run.smithi044.stdout: automake-1.16.2-8.el9.noarch 2024-04-22T06:30:06.198 INFO:teuthology.orchestra.run.smithi044.stdout: bison-3.7.4-5.el9.x86_64 2024-04-22T06:30:06.198 INFO:teuthology.orchestra.run.smithi044.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-22T06:30:06.199 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:30:06.199 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-base-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:30:06.199 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:30:06.199 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:30:06.199 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-grafana-dashboards-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T06:30:06.199 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-immutable-object-cache-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:30:06.199 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mds-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:30:06.199 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:30:06.199 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T06:30:06.199 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-dashboard-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T06:30:06.199 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T06:30:06.199 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-modules-core-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T06:30:06.199 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-rook-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T06:30:06.200 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mon-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:30:06.200 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-osd-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:30:06.200 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-prometheus-alerts-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T06:30:06.200 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-radosgw-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:30:06.200 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-selinux-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:30:06.200 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-test-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:30:06.200 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-volume-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T06:30:06.200 INFO:teuthology.orchestra.run.smithi044.stdout: cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T06:30:06.200 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:30:06.200 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-shell-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:30:06.200 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-top-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T06:30:06.200 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-22T06:30:06.200 INFO:teuthology.orchestra.run.smithi044.stdout: flex-2.6.4-9.el9.x86_64 2024-04-22T06:30:06.201 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-22T06:30:06.201 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-22T06:30:06.201 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-22T06:30:06.201 INFO:teuthology.orchestra.run.smithi044.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-22T06:30:06.201 INFO:teuthology.orchestra.run.smithi044.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-22T06:30:06.201 INFO:teuthology.orchestra.run.smithi044.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-22T06:30:06.201 INFO:teuthology.orchestra.run.smithi044.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-22T06:30:06.201 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-22T06:30:06.201 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-22T06:30:06.201 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs-devel-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:30:06.201 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs2-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:30:06.201 INFO:teuthology.orchestra.run.smithi044.stdout: libcephsqlite-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:30:06.201 INFO:teuthology.orchestra.run.smithi044.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-22T06:30:06.202 INFO:teuthology.orchestra.run.smithi044.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-22T06:30:06.202 INFO:teuthology.orchestra.run.smithi044.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-22T06:30:06.202 INFO:teuthology.orchestra.run.smithi044.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-22T06:30:06.202 INFO:teuthology.orchestra.run.smithi044.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-22T06:30:06.202 INFO:teuthology.orchestra.run.smithi044.stdout: librados-devel-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:30:06.202 INFO:teuthology.orchestra.run.smithi044.stdout: libradosstriper1-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:30:06.202 INFO:teuthology.orchestra.run.smithi044.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-22T06:30:06.202 INFO:teuthology.orchestra.run.smithi044.stdout: librgw2-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:30:06.202 INFO:teuthology.orchestra.run.smithi044.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-22T06:30:06.202 INFO:teuthology.orchestra.run.smithi044.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-22T06:30:06.202 INFO:teuthology.orchestra.run.smithi044.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-22T06:30:06.202 INFO:teuthology.orchestra.run.smithi044.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-22T06:30:06.202 INFO:teuthology.orchestra.run.smithi044.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-22T06:30:06.202 INFO:teuthology.orchestra.run.smithi044.stdout: lua-5.4.4-4.el9.x86_64 2024-04-22T06:30:06.203 INFO:teuthology.orchestra.run.smithi044.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-22T06:30:06.203 INFO:teuthology.orchestra.run.smithi044.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-22T06:30:06.203 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-22T06:30:06.203 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-22T06:30:06.203 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-22T06:30:06.203 INFO:teuthology.orchestra.run.smithi044.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-22T06:30:06.203 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-13.14-1.el9.x86_64 2024-04-22T06:30:06.203 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-22T06:30:06.203 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-22T06:30:06.203 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-22T06:30:06.203 INFO:teuthology.orchestra.run.smithi044.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-22T06:30:06.203 INFO:teuthology.orchestra.run.smithi044.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-22T06:30:06.203 INFO:teuthology.orchestra.run.smithi044.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-22T06:30:06.204 INFO:teuthology.orchestra.run.smithi044.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-22T06:30:06.204 INFO:teuthology.orchestra.run.smithi044.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-22T06:30:06.204 INFO:teuthology.orchestra.run.smithi044.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-22T06:30:06.204 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-22T06:30:06.204 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-argparse-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:30:06.204 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:30:06.204 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cephfs-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:30:06.204 INFO:teuthology.orchestra.run.smithi044.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-22T06:30:06.204 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-22T06:30:06.204 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-22T06:30:06.205 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-22T06:30:06.205 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-22T06:30:06.205 INFO:teuthology.orchestra.run.smithi044.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-22T06:30:06.205 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-22T06:30:06.205 INFO:teuthology.orchestra.run.smithi044.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-22T06:30:06.205 INFO:teuthology.orchestra.run.smithi044.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-22T06:30:06.205 INFO:teuthology.orchestra.run.smithi044.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-22T06:30:06.205 INFO:teuthology.orchestra.run.smithi044.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-22T06:30:06.205 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-22T06:30:06.205 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-22T06:30:06.205 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-22T06:30:06.205 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-22T06:30:06.206 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-22T06:30:06.206 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-22T06:30:06.206 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-22T06:30:06.206 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-22T06:30:06.206 INFO:teuthology.orchestra.run.smithi044.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-22T06:30:06.206 INFO:teuthology.orchestra.run.smithi044.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-22T06:30:06.206 INFO:teuthology.orchestra.run.smithi044.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-22T06:30:06.206 INFO:teuthology.orchestra.run.smithi044.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-22T06:30:06.206 INFO:teuthology.orchestra.run.smithi044.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-22T06:30:06.206 INFO:teuthology.orchestra.run.smithi044.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-22T06:30:06.206 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-22T06:30:06.206 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-22T06:30:06.206 INFO:teuthology.orchestra.run.smithi044.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-22T06:30:06.206 INFO:teuthology.orchestra.run.smithi044.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-22T06:30:06.207 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-22T06:30:06.207 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ply-3.11-14.el9.noarch 2024-04-22T06:30:06.207 INFO:teuthology.orchestra.run.smithi044.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-22T06:30:06.207 INFO:teuthology.orchestra.run.smithi044.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-22T06:30:06.207 INFO:teuthology.orchestra.run.smithi044.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-22T06:30:06.207 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-22T06:30:06.207 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-22T06:30:06.207 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-22T06:30:06.207 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-22T06:30:06.207 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-22T06:30:06.207 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-22T06:30:06.207 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rados-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:30:06.207 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rbd-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:30:06.207 INFO:teuthology.orchestra.run.smithi044.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-22T06:30:06.208 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-22T06:30:06.208 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-22T06:30:06.208 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rgw-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:30:06.208 INFO:teuthology.orchestra.run.smithi044.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-22T06:30:06.208 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-22T06:30:06.208 INFO:teuthology.orchestra.run.smithi044.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-22T06:30:06.208 INFO:teuthology.orchestra.run.smithi044.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-22T06:30:06.208 INFO:teuthology.orchestra.run.smithi044.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-22T06:30:06.208 INFO:teuthology.orchestra.run.smithi044.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-22T06:30:06.208 INFO:teuthology.orchestra.run.smithi044.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-22T06:30:06.208 INFO:teuthology.orchestra.run.smithi044.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-22T06:30:06.208 INFO:teuthology.orchestra.run.smithi044.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-22T06:30:06.208 INFO:teuthology.orchestra.run.smithi044.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-22T06:30:06.208 INFO:teuthology.orchestra.run.smithi044.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-22T06:30:06.209 INFO:teuthology.orchestra.run.smithi044.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-22T06:30:06.209 INFO:teuthology.orchestra.run.smithi044.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-22T06:30:06.209 INFO:teuthology.orchestra.run.smithi044.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-22T06:30:06.209 INFO:teuthology.orchestra.run.smithi044.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-22T06:30:06.209 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:30:06.209 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:30:06.209 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-nbd-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T06:30:06.209 INFO:teuthology.orchestra.run.smithi044.stdout: re2-1:20211101-3.el9.x86_64 2024-04-22T06:30:06.209 INFO:teuthology.orchestra.run.smithi044.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-22T06:30:06.210 INFO:teuthology.orchestra.run.smithi044.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-22T06:30:06.210 INFO:teuthology.orchestra.run.smithi044.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-22T06:30:06.210 INFO:teuthology.orchestra.run.smithi044.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-22T06:30:06.210 INFO:teuthology.orchestra.run.smithi044.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-22T06:30:06.211 INFO:teuthology.orchestra.run.smithi044.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-22T06:30:06.211 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:30:06.211 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-22T06:30:06.576 DEBUG:teuthology.parallel:result is None 2024-04-22T06:30:06.577 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=86f7587a5a09af35f0895e1a2d08527638fae697 2024-04-22T06:30:06.825 DEBUG:teuthology.orchestra.run.smithi044:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-22T06:30:06.847 INFO:teuthology.orchestra.run.smithi044.stdout:19.0.0-3156.g86f7587a.el9 2024-04-22T06:30:06.848 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.g86f7587a.el9 2024-04-22T06:30:06.848 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.g86f7587a is installed. 2024-04-22T06:30:06.849 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=86f7587a5a09af35f0895e1a2d08527638fae697 2024-04-22T06:30:07.003 DEBUG:teuthology.orchestra.run.smithi070:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-22T06:30:07.028 INFO:teuthology.orchestra.run.smithi070.stdout:19.0.0-3156.g86f7587a.el9 2024-04-22T06:30:07.029 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.g86f7587a.el9 2024-04-22T06:30:07.029 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.g86f7587a is installed. 2024-04-22T06:30:07.031 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-22T06:30:07.031 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T06:30:07.031 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-22T06:30:07.058 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T06:30:07.058 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-22T06:30:07.100 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-22T06:30:07.100 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T06:30:07.100 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/daemon-helper 2024-04-22T06:30:07.125 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-22T06:30:07.188 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T06:30:07.189 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/daemon-helper 2024-04-22T06:30:07.215 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-22T06:30:07.281 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-22T06:30:07.282 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T06:30:07.282 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-22T06:30:07.306 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-22T06:30:07.371 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T06:30:07.371 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-22T06:30:07.397 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-22T06:30:07.462 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-22T06:30:07.462 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T06:30:07.462 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/stdin-killer 2024-04-22T06:30:07.488 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-22T06:30:07.556 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T06:30:07.556 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/stdin-killer 2024-04-22T06:30:07.580 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-22T06:30:07.644 INFO:teuthology.run_tasks:Running task ceph... 2024-04-22T06:30:07.747 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-22T06:30:07.747 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 777 /var/log/ceph 2024-04-22T06:30:07.750 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 777 /var/log/ceph 2024-04-22T06:30:07.773 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-22T06:30:07.773 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-22T06:30:07.814 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-22T06:30:07.837 INFO:tasks.ceph:Creating extra log directories... 2024-04-22T06:30:07.837 DEBUG:teuthology.orchestra.run.smithi044:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-22T06:30:07.878 DEBUG:teuthology.orchestra.run.smithi070:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-22T06:30:07.914 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-22T06:30:07.914 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-22T06:30:07.939 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-22T06:30:07.939 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T06:30:07.939 DEBUG:teuthology.orchestra.run.smithi044:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-22T06:30:07.957 DEBUG:teuthology.orchestra.run.smithi044:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-22T06:30:08.074 DEBUG:teuthology.orchestra.run.smithi044:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-22T06:30:08.099 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T06:30:08.099 DEBUG:teuthology.orchestra.run.smithi070:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-22T06:30:08.114 DEBUG:teuthology.orchestra.run.smithi070:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-22T06:30:08.220 DEBUG:teuthology.orchestra.run.smithi070:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-22T06:30:08.266 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-22T06:30:08.266 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-22T06:30:08.266 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668133', 'branch': 'wip-lusov-quiescer', 'description': 'fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/with-quiesce 2-workunit/fs/trivial_sync}}', 'email': None, 'first_in_suite': False, 'job_id': '7668133', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-lusov-quiescer'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum'], 'sha1': '86f7587a5a09af35f0895e1a2d08527638fae697'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'check-counter': {'counters': {'mds': ['mds.exported', 'mds.imported']}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '86f7587a5a09af35f0895e1a2d08527638fae697'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-lusov-quiescer', 'sha1': '86f7587a5a09af35f0895e1a2d08527638fae697'}}, 'owner': 'scheduled_leonidus@teuthology', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 4362, 'sha1': '86f7587a5a09af35f0895e1a2d08527638fae697', 'sleep_before_teardown': 0, 'subset': '222/111111', 'suite': 'fs', 'suite_branch': 'wip-lusov-quiescer', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '86f7587a5a09af35f0895e1a2d08527638fae697', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi044.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOkhDsKQt1CCnIug9tZrc1CemwV6jDC9sr/9o9GE8Hsi+FL16G3gngvPEsR3q4ADneLtalMZOZq8CmVR7TIRdeA=', 'smithi070.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEcso0amEbD7LWkysbF+kOSM3IVibZdDd54bL/6XuJSZ7MZk97LGx/Xh4KeuLIRyxLqN83Xjbno3yb2y3nWBpwk='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '86f7587a5a09af35f0895e1a2d08527638fae697'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum'], 'sha1': '86f7587a5a09af35f0895e1a2d08527638fae697', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'quiescer': None}, {'workunit': {'clients': {'all': ['fs/misc/trivial_sync.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-22_05:52:53', 'tube': 'smithi', 'user': 'leonidus', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-22T06:30:08.267 DEBUG:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-22T06:30:08.269 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-22T06:30:08.314 DEBUG:teuthology.orchestra.run.smithi044:> sudo install -d -m0777 -- /var/run/ceph 2024-04-22T06:30:08.325 DEBUG:teuthology.orchestra.run.smithi070:> sudo install -d -m0777 -- /var/run/ceph 2024-04-22T06:30:08.377 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T06:30:08.378 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/scratch_devs of=/dev/stdout 2024-04-22T06:30:08.404 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-22T06:30:08.404 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_1 2024-04-22T06:30:08.460 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-22T06:30:08.460 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T06:30:08.460 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-22T06:30:08.461 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T06:30:08.461 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T06:30:08.461 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-04-22 06:29:58.830359348 +0000 2024-04-22T06:30:08.461 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-04-22 06:27:47.690396576 +0000 2024-04-22T06:30:08.461 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-04-22 06:27:47.690396576 +0000 2024-04-22T06:30:08.461 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-04-22 06:27:47.690396576 +0000 2024-04-22T06:30:08.461 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-22T06:30:08.524 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-04-22T06:30:08.524 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-04-22T06:30:08.524 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000264393 s, 1.9 MB/s 2024-04-22T06:30:08.525 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-22T06:30:08.582 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_2 2024-04-22T06:30:08.638 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-22T06:30:08.638 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T06:30:08.638 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-22T06:30:08.638 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T06:30:08.638 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T06:30:08.638 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-04-22 06:29:58.830359348 +0000 2024-04-22T06:30:08.638 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-04-22 06:27:48.050391105 +0000 2024-04-22T06:30:08.638 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-04-22 06:27:48.050391105 +0000 2024-04-22T06:30:08.638 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-04-22 06:27:48.050391105 +0000 2024-04-22T06:30:08.639 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-22T06:30:08.700 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-04-22T06:30:08.700 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-04-22T06:30:08.700 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.00021428 s, 2.4 MB/s 2024-04-22T06:30:08.702 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-22T06:30:08.759 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_3 2024-04-22T06:30:08.815 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-22T06:30:08.815 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T06:30:08.815 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-22T06:30:08.815 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T06:30:08.815 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T06:30:08.815 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-04-22 06:29:58.831359333 +0000 2024-04-22T06:30:08.815 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-04-22 06:27:48.401385772 +0000 2024-04-22T06:30:08.815 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-04-22 06:27:48.401385772 +0000 2024-04-22T06:30:08.816 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-04-22 06:27:48.401385772 +0000 2024-04-22T06:30:08.816 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-22T06:30:08.879 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-04-22T06:30:08.879 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-04-22T06:30:08.879 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000211886 s, 2.4 MB/s 2024-04-22T06:30:08.880 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-22T06:30:08.937 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_4 2024-04-22T06:30:08.993 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-22T06:30:08.993 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T06:30:08.993 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-22T06:30:08.993 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T06:30:08.993 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T06:30:08.993 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-04-22 06:29:58.831359333 +0000 2024-04-22T06:30:08.993 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-04-22 06:27:48.743380575 +0000 2024-04-22T06:30:08.993 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-04-22 06:27:48.743380575 +0000 2024-04-22T06:30:08.993 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-04-22 06:27:48.743380575 +0000 2024-04-22T06:30:08.994 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-22T06:30:09.056 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-04-22T06:30:09.056 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-04-22T06:30:09.056 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000206713 s, 2.5 MB/s 2024-04-22T06:30:09.057 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-22T06:30:09.114 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-22T06:30:09.114 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T06:30:09.114 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/scratch_devs of=/dev/stdout 2024-04-22T06:30:09.130 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-22T06:30:09.131 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_1 2024-04-22T06:30:09.187 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-22T06:30:09.187 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T06:30:09.187 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-22T06:30:09.187 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T06:30:09.187 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T06:30:09.187 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-04-22 06:29:38.680399649 +0000 2024-04-22T06:30:09.188 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-04-22 06:27:45.479032428 +0000 2024-04-22T06:30:09.188 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-04-22 06:27:45.479032428 +0000 2024-04-22T06:30:09.188 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-04-22 06:27:45.479032428 +0000 2024-04-22T06:30:09.188 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-22T06:30:09.250 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-04-22T06:30:09.250 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-04-22T06:30:09.250 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000123802 s, 4.1 MB/s 2024-04-22T06:30:09.252 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-22T06:30:09.309 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_2 2024-04-22T06:30:09.366 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-22T06:30:09.366 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T06:30:09.366 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-22T06:30:09.366 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T06:30:09.366 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T06:30:09.366 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-04-22 06:29:38.680399649 +0000 2024-04-22T06:30:09.366 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-04-22 06:27:45.806022698 +0000 2024-04-22T06:30:09.366 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-04-22 06:27:45.806022698 +0000 2024-04-22T06:30:09.366 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-04-22 06:27:45.806022698 +0000 2024-04-22T06:30:09.367 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-22T06:30:09.429 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-04-22T06:30:09.430 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-04-22T06:30:09.430 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000106618 s, 4.8 MB/s 2024-04-22T06:30:09.431 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-22T06:30:09.487 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_3 2024-04-22T06:30:09.542 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-22T06:30:09.542 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T06:30:09.542 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-22T06:30:09.543 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T06:30:09.543 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T06:30:09.543 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-04-22 06:29:38.681399620 +0000 2024-04-22T06:30:09.543 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-04-22 06:27:46.130013058 +0000 2024-04-22T06:30:09.543 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-04-22 06:27:46.130013058 +0000 2024-04-22T06:30:09.543 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-04-22 06:27:46.130013058 +0000 2024-04-22T06:30:09.543 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-22T06:30:09.604 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-04-22T06:30:09.604 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-04-22T06:30:09.605 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.00010495 s, 4.9 MB/s 2024-04-22T06:30:09.606 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-22T06:30:09.662 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_4 2024-04-22T06:30:09.716 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-22T06:30:09.716 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T06:30:09.717 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-22T06:30:09.717 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T06:30:09.717 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T06:30:09.717 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-04-22 06:29:38.681399620 +0000 2024-04-22T06:30:09.717 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-04-22 06:27:46.622998390 +0000 2024-04-22T06:30:09.717 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-04-22 06:27:46.622998390 +0000 2024-04-22T06:30:09.717 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-04-22 06:27:46.622998390 +0000 2024-04-22T06:30:09.717 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-22T06:30:09.779 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-04-22T06:30:09.780 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-04-22T06:30:09.780 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000100238 s, 5.1 MB/s 2024-04-22T06:30:09.780 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-22T06:30:09.835 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-22T06:30:09.835 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi044.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi070.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-22T06:30:09.835 INFO:tasks.ceph:Generating config... 2024-04-22T06:30:09.862 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-22T06:30:09.863 INFO:tasks.ceph:[client] client_shutdown_timeout = 120 2024-04-22T06:30:09.863 INFO:tasks.ceph:[client] debug client = 20 2024-04-22T06:30:09.863 INFO:tasks.ceph:[client] debug ms = 1 2024-04-22T06:30:09.863 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-22T06:30:09.863 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-22T06:30:09.863 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-22T06:30:09.863 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-22T06:30:09.863 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-22T06:30:09.863 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2024-04-22T06:30:09.863 INFO:tasks.ceph:[mds] mds bal merge size = 5 2024-04-22T06:30:09.863 INFO:tasks.ceph:[mds] mds bal split bits = 3 2024-04-22T06:30:09.863 INFO:tasks.ceph:[mds] mds bal split size = 100 2024-04-22T06:30:09.863 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-22T06:30:09.864 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-22T06:30:09.864 INFO:tasks.ceph:[mds] mds oft prefetch dirfrags = True 2024-04-22T06:30:09.864 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-22T06:30:09.864 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-22T06:30:09.864 INFO:tasks.ceph:[mds] mds_dir_prefetch = True 2024-04-22T06:30:09.864 INFO:tasks.ceph:[mds] mds_sleep_rank_change = 5000000.0 2024-04-22T06:30:09.864 INFO:tasks.ceph:[mds] mon client directed command retry = 5 2024-04-22T06:30:09.864 INFO:tasks.ceph:[mds] ms inject delay max = 1 2024-04-22T06:30:09.864 INFO:tasks.ceph:[mds] ms inject delay probability = 0.005 2024-04-22T06:30:09.864 INFO:tasks.ceph:[mds] ms inject delay type = client mds osd 2024-04-22T06:30:09.864 INFO:tasks.ceph:[mds] ms inject socket failures = 2500 2024-04-22T06:30:09.864 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-22T06:30:09.864 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-22T06:30:09.864 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-22T06:30:09.864 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-22T06:30:09.865 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-22T06:30:09.865 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-22T06:30:09.865 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-22T06:30:09.865 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-22T06:30:09.865 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-22T06:30:09.865 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-22T06:30:09.865 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-22T06:30:09.865 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-22T06:30:09.865 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-22T06:30:09.865 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-22T06:30:09.865 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-22T06:30:09.865 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-22T06:30:09.865 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-22T06:30:09.865 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-22T06:30:09.866 INFO:tasks.ceph:[osd] mon client directed command retry = 5 2024-04-22T06:30:09.866 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-22T06:30:09.866 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-22T06:30:09.866 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-22T06:30:09.866 INFO:tasks.ceph:[osd] ms inject delay max = 1 2024-04-22T06:30:09.866 INFO:tasks.ceph:[osd] ms inject delay probability = 0.005 2024-04-22T06:30:09.866 INFO:tasks.ceph:[osd] ms inject delay type = client mds 2024-04-22T06:30:09.866 INFO:tasks.ceph:[osd] ms inject socket failures = 2500 2024-04-22T06:30:09.866 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-04-22T06:30:09.866 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-22T06:30:09.866 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-04-22T06:30:09.866 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-22T06:30:09.866 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-22T06:30:09.867 INFO:tasks.ceph:Setting up mon.a... 2024-04-22T06:30:09.867 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-22T06:30:09.924 INFO:teuthology.orchestra.run.smithi044.stdout:creating /etc/ceph/ceph.keyring 2024-04-22T06:30:09.926 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-22T06:30:09.970 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-22T06:30:10.036 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.44'), ('mon.b', '172.21.15.70'), ('mon.c', '[v2:172.21.15.70:3301,v1:172.21.15.70:6790]')] 2024-04-22T06:30:10.036 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.44,172.21.15.70,[v2:172.21.15.70:3301,v1:172.21.15.70:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.e': {}, 'mds.b': {}, 'mds.d': {}, 'mds.f': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-22T06:30:10.037 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T06:30:10.037 DEBUG:teuthology.orchestra.run.smithi044:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-22T06:30:10.092 DEBUG:teuthology.orchestra.run.smithi044:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.44 --add b 172.21.15.70 --addv c '[v2:172.21.15.70:3301,v1:172.21.15.70:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-22T06:30:10.180 INFO:teuthology.orchestra.run.smithi044.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-22T06:30:10.180 INFO:teuthology.orchestra.run.smithi044.stdout:monmaptool: generated fsid 9b0a34ca-403d-4513-8a66-d3749ba559aa 2024-04-22T06:30:10.180 INFO:teuthology.orchestra.run.smithi044.stdout:setting min_mon_release = quincy 2024-04-22T06:30:10.180 INFO:teuthology.orchestra.run.smithi044.stdout:epoch 0 2024-04-22T06:30:10.180 INFO:teuthology.orchestra.run.smithi044.stdout:fsid 9b0a34ca-403d-4513-8a66-d3749ba559aa 2024-04-22T06:30:10.180 INFO:teuthology.orchestra.run.smithi044.stdout:last_changed 2024-04-22T06:30:10.182106+0000 2024-04-22T06:30:10.180 INFO:teuthology.orchestra.run.smithi044.stdout:created 2024-04-22T06:30:10.182106+0000 2024-04-22T06:30:10.181 INFO:teuthology.orchestra.run.smithi044.stdout:min_mon_release 17 (quincy) 2024-04-22T06:30:10.181 INFO:teuthology.orchestra.run.smithi044.stdout:election_strategy: 1 2024-04-22T06:30:10.181 INFO:teuthology.orchestra.run.smithi044.stdout:0: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.a 2024-04-22T06:30:10.181 INFO:teuthology.orchestra.run.smithi044.stdout:1: [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] mon.b 2024-04-22T06:30:10.181 INFO:teuthology.orchestra.run.smithi044.stdout:2: [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] mon.c 2024-04-22T06:30:10.181 INFO:teuthology.orchestra.run.smithi044.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-22T06:30:10.182 DEBUG:teuthology.orchestra.run.smithi044:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-22T06:30:10.197 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 9b0a34ca-403d-4513-8a66-d3749ba559aa... 2024-04-22T06:30:10.198 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-22T06:30:10.240 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-22T06:30:10.279 INFO:teuthology.orchestra.run.smithi044.stdout:[global] 2024-04-22T06:30:10.280 INFO:teuthology.orchestra.run.smithi044.stdout: chdir = "" 2024-04-22T06:30:10.280 INFO:teuthology.orchestra.run.smithi044.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-22T06:30:10.280 INFO:teuthology.orchestra.run.smithi044.stdout: auth supported = cephx 2024-04-22T06:30:10.280 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:30:10.280 INFO:teuthology.orchestra.run.smithi044.stdout: filestore xattr use omap = true 2024-04-22T06:30:10.280 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:30:10.280 INFO:teuthology.orchestra.run.smithi044.stdout: mon clock drift allowed = 1.000 2024-04-22T06:30:10.280 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:30:10.280 INFO:teuthology.orchestra.run.smithi044.stdout: osd crush chooseleaf type = 0 2024-04-22T06:30:10.280 INFO:teuthology.orchestra.run.smithi044.stdout: auth debug = true 2024-04-22T06:30:10.280 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:30:10.280 INFO:teuthology.orchestra.run.smithi044.stdout: ms die on old message = true 2024-04-22T06:30:10.281 INFO:teuthology.orchestra.run.smithi044.stdout: ms die on bug = true 2024-04-22T06:30:10.281 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:30:10.281 INFO:teuthology.orchestra.run.smithi044.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-22T06:30:10.281 INFO:teuthology.orchestra.run.smithi044.stdout: mon pg warn max object skew = 0 2024-04-22T06:30:10.281 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:30:10.281 INFO:teuthology.orchestra.run.smithi044.stdout: # disable pg_autoscaler by default for new pools 2024-04-22T06:30:10.281 INFO:teuthology.orchestra.run.smithi044.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-22T06:30:10.281 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:30:10.281 INFO:teuthology.orchestra.run.smithi044.stdout: osd pool default size = 2 2024-04-22T06:30:10.281 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:30:10.281 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd allow primary affinity = true 2024-04-22T06:30:10.281 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd allow pg remap = true 2024-04-22T06:30:10.281 INFO:teuthology.orchestra.run.smithi044.stdout: mon warn on legacy crush tunables = false 2024-04-22T06:30:10.282 INFO:teuthology.orchestra.run.smithi044.stdout: mon warn on crush straw calc version zero = false 2024-04-22T06:30:10.282 INFO:teuthology.orchestra.run.smithi044.stdout: mon warn on no sortbitwise = false 2024-04-22T06:30:10.282 INFO:teuthology.orchestra.run.smithi044.stdout: mon warn on osd down out interval zero = false 2024-04-22T06:30:10.282 INFO:teuthology.orchestra.run.smithi044.stdout: mon warn on too few osds = false 2024-04-22T06:30:10.282 INFO:teuthology.orchestra.run.smithi044.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-22T06:30:10.282 INFO:teuthology.orchestra.run.smithi044.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-22T06:30:10.282 INFO:teuthology.orchestra.run.smithi044.stdout: mon_allow_pool_size_one = true 2024-04-22T06:30:10.282 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:30:10.282 INFO:teuthology.orchestra.run.smithi044.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-22T06:30:10.282 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:30:10.282 INFO:teuthology.orchestra.run.smithi044.stdout: osd default data pool replay window = 5 2024-04-22T06:30:10.282 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:30:10.283 INFO:teuthology.orchestra.run.smithi044.stdout: mon allow pool delete = true 2024-04-22T06:30:10.283 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:30:10.283 INFO:teuthology.orchestra.run.smithi044.stdout: mon cluster log file level = debug 2024-04-22T06:30:10.283 INFO:teuthology.orchestra.run.smithi044.stdout: debug asserts on shutdown = true 2024-04-22T06:30:10.283 INFO:teuthology.orchestra.run.smithi044.stdout: mon health detail to clog = false 2024-04-22T06:30:10.283 INFO:teuthology.orchestra.run.smithi044.stdout: mon host = "172.21.15.44,172.21.15.70,[v2:172.21.15.70:3301,v1:172.21.15.70:6790]" 2024-04-22T06:30:10.283 INFO:teuthology.orchestra.run.smithi044.stdout: fsid = 9b0a34ca-403d-4513-8a66-d3749ba559aa 2024-04-22T06:30:10.283 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:30:10.283 INFO:teuthology.orchestra.run.smithi044.stdout:[osd] 2024-04-22T06:30:10.283 INFO:teuthology.orchestra.run.smithi044.stdout: osd journal size = 100 2024-04-22T06:30:10.283 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:30:10.283 INFO:teuthology.orchestra.run.smithi044.stdout: osd scrub load threshold = 5.0 2024-04-22T06:30:10.283 INFO:teuthology.orchestra.run.smithi044.stdout: osd scrub max interval = 600 2024-04-22T06:30:10.284 INFO:teuthology.orchestra.run.smithi044.stdout: osd mclock profile = high_recovery_ops 2024-04-22T06:30:10.284 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:30:10.284 INFO:teuthology.orchestra.run.smithi044.stdout: osd recover clone overlap = true 2024-04-22T06:30:10.284 INFO:teuthology.orchestra.run.smithi044.stdout: osd recovery max chunk = 1048576 2024-04-22T06:30:10.284 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:30:10.284 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug shutdown = true 2024-04-22T06:30:10.284 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug op order = true 2024-04-22T06:30:10.284 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug verify stray on activate = true 2024-04-22T06:30:10.284 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug trim objects = true 2024-04-22T06:30:10.284 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:30:10.284 INFO:teuthology.orchestra.run.smithi044.stdout: osd open classes on start = true 2024-04-22T06:30:10.284 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug pg log writeout = true 2024-04-22T06:30:10.284 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:30:10.285 INFO:teuthology.orchestra.run.smithi044.stdout: osd deep scrub update digest min age = 30 2024-04-22T06:30:10.285 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:30:10.285 INFO:teuthology.orchestra.run.smithi044.stdout: osd map max advance = 10 2024-04-22T06:30:10.285 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:30:10.285 INFO:teuthology.orchestra.run.smithi044.stdout: journal zero on create = true 2024-04-22T06:30:10.285 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:30:10.285 INFO:teuthology.orchestra.run.smithi044.stdout: filestore ondisk finisher threads = 3 2024-04-22T06:30:10.285 INFO:teuthology.orchestra.run.smithi044.stdout: filestore apply finisher threads = 3 2024-04-22T06:30:10.285 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:30:10.285 INFO:teuthology.orchestra.run.smithi044.stdout: bdev debug aio = true 2024-04-22T06:30:10.285 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug misdirected ops = true 2024-04-22T06:30:10.285 INFO:teuthology.orchestra.run.smithi044.stdout: bluestore block size = 96636764160 2024-04-22T06:30:10.286 INFO:teuthology.orchestra.run.smithi044.stdout: bluestore fsck on mount = True 2024-04-22T06:30:10.286 INFO:teuthology.orchestra.run.smithi044.stdout: debug bluefs = 20 2024-04-22T06:30:10.286 INFO:teuthology.orchestra.run.smithi044.stdout: debug bluestore = 20 2024-04-22T06:30:10.286 INFO:teuthology.orchestra.run.smithi044.stdout: debug ms = 1 2024-04-22T06:30:10.286 INFO:teuthology.orchestra.run.smithi044.stdout: debug osd = 20 2024-04-22T06:30:10.286 INFO:teuthology.orchestra.run.smithi044.stdout: debug rocksdb = 10 2024-04-22T06:30:10.286 INFO:teuthology.orchestra.run.smithi044.stdout: mon client directed command retry = 5 2024-04-22T06:30:10.286 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd backfillfull_ratio = 0.85 2024-04-22T06:30:10.286 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd full ratio = 0.9 2024-04-22T06:30:10.286 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd nearfull ratio = 0.8 2024-04-22T06:30:10.286 INFO:teuthology.orchestra.run.smithi044.stdout: ms inject delay max = 1 2024-04-22T06:30:10.286 INFO:teuthology.orchestra.run.smithi044.stdout: ms inject delay probability = 0.005 2024-04-22T06:30:10.286 INFO:teuthology.orchestra.run.smithi044.stdout: ms inject delay type = client mds 2024-04-22T06:30:10.287 INFO:teuthology.orchestra.run.smithi044.stdout: ms inject socket failures = 2500 2024-04-22T06:30:10.287 INFO:teuthology.orchestra.run.smithi044.stdout: osd blocked scrub grace period = 3600 2024-04-22T06:30:10.287 INFO:teuthology.orchestra.run.smithi044.stdout: osd failsafe full ratio = 0.95 2024-04-22T06:30:10.287 INFO:teuthology.orchestra.run.smithi044.stdout: osd max markdown count = 1000 2024-04-22T06:30:10.287 INFO:teuthology.orchestra.run.smithi044.stdout: osd objectstore = bluestore 2024-04-22T06:30:10.287 INFO:teuthology.orchestra.run.smithi044.stdout: osd op complaint time = 180 2024-04-22T06:30:10.287 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:30:10.287 INFO:teuthology.orchestra.run.smithi044.stdout:[mgr] 2024-04-22T06:30:10.287 INFO:teuthology.orchestra.run.smithi044.stdout: debug ms = 1 2024-04-22T06:30:10.287 INFO:teuthology.orchestra.run.smithi044.stdout: debug mgr = 20 2024-04-22T06:30:10.287 INFO:teuthology.orchestra.run.smithi044.stdout: debug mon = 20 2024-04-22T06:30:10.287 INFO:teuthology.orchestra.run.smithi044.stdout: debug auth = 20 2024-04-22T06:30:10.287 INFO:teuthology.orchestra.run.smithi044.stdout: mon reweight min pgs per osd = 4 2024-04-22T06:30:10.287 INFO:teuthology.orchestra.run.smithi044.stdout: mon reweight min bytes per osd = 10 2024-04-22T06:30:10.288 INFO:teuthology.orchestra.run.smithi044.stdout: mgr/telemetry/nag = false 2024-04-22T06:30:10.288 INFO:teuthology.orchestra.run.smithi044.stdout: debug client = 20 2024-04-22T06:30:10.288 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:30:10.288 INFO:teuthology.orchestra.run.smithi044.stdout:[mon] 2024-04-22T06:30:10.288 INFO:teuthology.orchestra.run.smithi044.stdout: debug ms = 1 2024-04-22T06:30:10.288 INFO:teuthology.orchestra.run.smithi044.stdout: debug mon = 20 2024-04-22T06:30:10.288 INFO:teuthology.orchestra.run.smithi044.stdout: debug paxos = 20 2024-04-22T06:30:10.288 INFO:teuthology.orchestra.run.smithi044.stdout: debug auth = 20 2024-04-22T06:30:10.288 INFO:teuthology.orchestra.run.smithi044.stdout: mon data avail warn = 5 2024-04-22T06:30:10.288 INFO:teuthology.orchestra.run.smithi044.stdout: mon mgr mkfs grace = 240 2024-04-22T06:30:10.288 INFO:teuthology.orchestra.run.smithi044.stdout: mon reweight min pgs per osd = 4 2024-04-22T06:30:10.288 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd reporter subtree level = osd 2024-04-22T06:30:10.288 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd prime pg temp = true 2024-04-22T06:30:10.289 INFO:teuthology.orchestra.run.smithi044.stdout: mon reweight min bytes per osd = 10 2024-04-22T06:30:10.289 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:30:10.289 INFO:teuthology.orchestra.run.smithi044.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-22T06:30:10.289 INFO:teuthology.orchestra.run.smithi044.stdout: auth mon ticket ttl = 660 # 11m 2024-04-22T06:30:10.289 INFO:teuthology.orchestra.run.smithi044.stdout: auth service ticket ttl = 240 # 4m 2024-04-22T06:30:10.289 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:30:10.289 INFO:teuthology.orchestra.run.smithi044.stdout: # don't complain about insecure global_id in the test suite 2024-04-22T06:30:10.289 INFO:teuthology.orchestra.run.smithi044.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-22T06:30:10.289 INFO:teuthology.orchestra.run.smithi044.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-22T06:30:10.289 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:30:10.289 INFO:teuthology.orchestra.run.smithi044.stdout: # 1m isn't quite enough 2024-04-22T06:30:10.289 INFO:teuthology.orchestra.run.smithi044.stdout: mon_down_mkfs_grace = 2m 2024-04-22T06:30:10.289 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:30:10.289 INFO:teuthology.orchestra.run.smithi044.stdout: mon_warn_on_filestore_osds = false 2024-04-22T06:30:10.289 INFO:teuthology.orchestra.run.smithi044.stdout: mon down mkfs grace = 300 2024-04-22T06:30:10.290 INFO:teuthology.orchestra.run.smithi044.stdout: mon op complaint time = 120 2024-04-22T06:30:10.290 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:30:10.290 INFO:teuthology.orchestra.run.smithi044.stdout:[client] 2024-04-22T06:30:10.290 INFO:teuthology.orchestra.run.smithi044.stdout: rgw cache enabled = true 2024-04-22T06:30:10.290 INFO:teuthology.orchestra.run.smithi044.stdout: rgw enable ops log = true 2024-04-22T06:30:10.290 INFO:teuthology.orchestra.run.smithi044.stdout: rgw enable usage log = true 2024-04-22T06:30:10.290 INFO:teuthology.orchestra.run.smithi044.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-22T06:30:10.290 INFO:teuthology.orchestra.run.smithi044.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-22T06:30:10.290 INFO:teuthology.orchestra.run.smithi044.stdout: client mount timeout = 600 2024-04-22T06:30:10.290 INFO:teuthology.orchestra.run.smithi044.stdout: client_shutdown_timeout = 120 2024-04-22T06:30:10.290 INFO:teuthology.orchestra.run.smithi044.stdout: debug client = 20 2024-04-22T06:30:10.291 INFO:teuthology.orchestra.run.smithi044.stdout: debug ms = 1 2024-04-22T06:30:10.291 INFO:teuthology.orchestra.run.smithi044.stdout: rados mon op timeout = 900 2024-04-22T06:30:10.291 INFO:teuthology.orchestra.run.smithi044.stdout: rados osd op timeout = 900 2024-04-22T06:30:10.291 INFO:teuthology.orchestra.run.smithi044.stdout:[mon.a] 2024-04-22T06:30:10.291 INFO:teuthology.orchestra.run.smithi044.stdout:[mon.b] 2024-04-22T06:30:10.291 INFO:teuthology.orchestra.run.smithi044.stdout:[mon.c] 2024-04-22T06:30:10.291 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.a] 2024-04-22T06:30:10.291 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.c] 2024-04-22T06:30:10.291 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.e] 2024-04-22T06:30:10.291 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.b] 2024-04-22T06:30:10.291 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.d] 2024-04-22T06:30:10.291 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.f] 2024-04-22T06:30:10.291 INFO:teuthology.orchestra.run.smithi044.stdout:[mds] 2024-04-22T06:30:10.291 INFO:teuthology.orchestra.run.smithi044.stdout: debug mds = 20 2024-04-22T06:30:10.292 INFO:teuthology.orchestra.run.smithi044.stdout: debug mds balancer = 20 2024-04-22T06:30:10.292 INFO:teuthology.orchestra.run.smithi044.stdout: debug ms = 1 2024-04-22T06:30:10.292 INFO:teuthology.orchestra.run.smithi044.stdout: mds bal fragment size max = 10000 2024-04-22T06:30:10.292 INFO:teuthology.orchestra.run.smithi044.stdout: mds bal merge size = 5 2024-04-22T06:30:10.292 INFO:teuthology.orchestra.run.smithi044.stdout: mds bal split bits = 3 2024-04-22T06:30:10.292 INFO:teuthology.orchestra.run.smithi044.stdout: mds bal split size = 100 2024-04-22T06:30:10.292 INFO:teuthology.orchestra.run.smithi044.stdout: mds debug frag = True 2024-04-22T06:30:10.292 INFO:teuthology.orchestra.run.smithi044.stdout: mds debug scatterstat = True 2024-04-22T06:30:10.292 INFO:teuthology.orchestra.run.smithi044.stdout: mds oft prefetch dirfrags = True 2024-04-22T06:30:10.292 INFO:teuthology.orchestra.run.smithi044.stdout: mds op complaint time = 180 2024-04-22T06:30:10.292 INFO:teuthology.orchestra.run.smithi044.stdout: mds verify scatter = True 2024-04-22T06:30:10.292 INFO:teuthology.orchestra.run.smithi044.stdout: mds_dir_prefetch = True 2024-04-22T06:30:10.292 INFO:teuthology.orchestra.run.smithi044.stdout: mds_sleep_rank_change = 5000000.0 2024-04-22T06:30:10.292 INFO:teuthology.orchestra.run.smithi044.stdout: mon client directed command retry = 5 2024-04-22T06:30:10.292 INFO:teuthology.orchestra.run.smithi044.stdout: ms inject delay max = 1 2024-04-22T06:30:10.292 INFO:teuthology.orchestra.run.smithi044.stdout: ms inject delay probability = 0.005 2024-04-22T06:30:10.293 INFO:teuthology.orchestra.run.smithi044.stdout: ms inject delay type = client mds osd 2024-04-22T06:30:10.293 INFO:teuthology.orchestra.run.smithi044.stdout: ms inject socket failures = 2500 2024-04-22T06:30:10.293 INFO:teuthology.orchestra.run.smithi044.stdout: osd op complaint time = 180 2024-04-22T06:30:10.293 INFO:teuthology.orchestra.run.smithi044.stdout: rados mon op timeout = 900 2024-04-22T06:30:10.293 INFO:teuthology.orchestra.run.smithi044.stdout: rados osd op timeout = 900 2024-04-22T06:30:10.294 INFO:teuthology.orchestra.run.smithi070.stdout:[global] 2024-04-22T06:30:10.294 INFO:teuthology.orchestra.run.smithi070.stdout: chdir = "" 2024-04-22T06:30:10.295 INFO:teuthology.orchestra.run.smithi070.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-22T06:30:10.295 INFO:teuthology.orchestra.run.smithi070.stdout: auth supported = cephx 2024-04-22T06:30:10.295 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:30:10.295 INFO:teuthology.orchestra.run.smithi070.stdout: filestore xattr use omap = true 2024-04-22T06:30:10.295 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:30:10.295 INFO:teuthology.orchestra.run.smithi070.stdout: mon clock drift allowed = 1.000 2024-04-22T06:30:10.295 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:30:10.295 INFO:teuthology.orchestra.run.smithi070.stdout: osd crush chooseleaf type = 0 2024-04-22T06:30:10.295 INFO:teuthology.orchestra.run.smithi070.stdout: auth debug = true 2024-04-22T06:30:10.295 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:30:10.295 INFO:teuthology.orchestra.run.smithi070.stdout: ms die on old message = true 2024-04-22T06:30:10.295 INFO:teuthology.orchestra.run.smithi070.stdout: ms die on bug = true 2024-04-22T06:30:10.295 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:30:10.295 INFO:teuthology.orchestra.run.smithi070.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-22T06:30:10.296 INFO:teuthology.orchestra.run.smithi070.stdout: mon pg warn max object skew = 0 2024-04-22T06:30:10.296 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:30:10.296 INFO:teuthology.orchestra.run.smithi070.stdout: # disable pg_autoscaler by default for new pools 2024-04-22T06:30:10.296 INFO:teuthology.orchestra.run.smithi070.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-22T06:30:10.296 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:30:10.296 INFO:teuthology.orchestra.run.smithi070.stdout: osd pool default size = 2 2024-04-22T06:30:10.296 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:30:10.296 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd allow primary affinity = true 2024-04-22T06:30:10.296 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd allow pg remap = true 2024-04-22T06:30:10.296 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on legacy crush tunables = false 2024-04-22T06:30:10.296 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on crush straw calc version zero = false 2024-04-22T06:30:10.296 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on no sortbitwise = false 2024-04-22T06:30:10.296 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on osd down out interval zero = false 2024-04-22T06:30:10.296 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on too few osds = false 2024-04-22T06:30:10.296 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-22T06:30:10.296 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-22T06:30:10.297 INFO:teuthology.orchestra.run.smithi070.stdout: mon_allow_pool_size_one = true 2024-04-22T06:30:10.297 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:30:10.297 INFO:teuthology.orchestra.run.smithi070.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-22T06:30:10.297 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:30:10.297 INFO:teuthology.orchestra.run.smithi070.stdout: osd default data pool replay window = 5 2024-04-22T06:30:10.297 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:30:10.297 INFO:teuthology.orchestra.run.smithi070.stdout: mon allow pool delete = true 2024-04-22T06:30:10.297 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:30:10.297 INFO:teuthology.orchestra.run.smithi070.stdout: mon cluster log file level = debug 2024-04-22T06:30:10.297 INFO:teuthology.orchestra.run.smithi070.stdout: debug asserts on shutdown = true 2024-04-22T06:30:10.297 INFO:teuthology.orchestra.run.smithi070.stdout: mon health detail to clog = false 2024-04-22T06:30:10.297 INFO:teuthology.orchestra.run.smithi070.stdout: mon host = "172.21.15.44,172.21.15.70,[v2:172.21.15.70:3301,v1:172.21.15.70:6790]" 2024-04-22T06:30:10.297 INFO:teuthology.orchestra.run.smithi070.stdout: fsid = 9b0a34ca-403d-4513-8a66-d3749ba559aa 2024-04-22T06:30:10.297 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:30:10.297 INFO:teuthology.orchestra.run.smithi070.stdout:[osd] 2024-04-22T06:30:10.298 INFO:teuthology.orchestra.run.smithi070.stdout: osd journal size = 100 2024-04-22T06:30:10.298 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:30:10.298 INFO:teuthology.orchestra.run.smithi070.stdout: osd scrub load threshold = 5.0 2024-04-22T06:30:10.298 INFO:teuthology.orchestra.run.smithi070.stdout: osd scrub max interval = 600 2024-04-22T06:30:10.298 INFO:teuthology.orchestra.run.smithi070.stdout: osd mclock profile = high_recovery_ops 2024-04-22T06:30:10.298 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:30:10.298 INFO:teuthology.orchestra.run.smithi070.stdout: osd recover clone overlap = true 2024-04-22T06:30:10.298 INFO:teuthology.orchestra.run.smithi070.stdout: osd recovery max chunk = 1048576 2024-04-22T06:30:10.298 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:30:10.298 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug shutdown = true 2024-04-22T06:30:10.298 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug op order = true 2024-04-22T06:30:10.298 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug verify stray on activate = true 2024-04-22T06:30:10.299 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug trim objects = true 2024-04-22T06:30:10.299 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:30:10.299 INFO:teuthology.orchestra.run.smithi070.stdout: osd open classes on start = true 2024-04-22T06:30:10.299 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug pg log writeout = true 2024-04-22T06:30:10.299 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:30:10.299 INFO:teuthology.orchestra.run.smithi070.stdout: osd deep scrub update digest min age = 30 2024-04-22T06:30:10.299 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:30:10.299 INFO:teuthology.orchestra.run.smithi070.stdout: osd map max advance = 10 2024-04-22T06:30:10.299 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:30:10.299 INFO:teuthology.orchestra.run.smithi070.stdout: journal zero on create = true 2024-04-22T06:30:10.299 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:30:10.299 INFO:teuthology.orchestra.run.smithi070.stdout: filestore ondisk finisher threads = 3 2024-04-22T06:30:10.299 INFO:teuthology.orchestra.run.smithi070.stdout: filestore apply finisher threads = 3 2024-04-22T06:30:10.299 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:30:10.299 INFO:teuthology.orchestra.run.smithi070.stdout: bdev debug aio = true 2024-04-22T06:30:10.300 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug misdirected ops = true 2024-04-22T06:30:10.300 INFO:teuthology.orchestra.run.smithi070.stdout: bluestore block size = 96636764160 2024-04-22T06:30:10.300 INFO:teuthology.orchestra.run.smithi070.stdout: bluestore fsck on mount = True 2024-04-22T06:30:10.300 INFO:teuthology.orchestra.run.smithi070.stdout: debug bluefs = 20 2024-04-22T06:30:10.300 INFO:teuthology.orchestra.run.smithi070.stdout: debug bluestore = 20 2024-04-22T06:30:10.300 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2024-04-22T06:30:10.300 INFO:teuthology.orchestra.run.smithi070.stdout: debug osd = 20 2024-04-22T06:30:10.300 INFO:teuthology.orchestra.run.smithi070.stdout: debug rocksdb = 10 2024-04-22T06:30:10.300 INFO:teuthology.orchestra.run.smithi070.stdout: mon client directed command retry = 5 2024-04-22T06:30:10.300 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd backfillfull_ratio = 0.85 2024-04-22T06:30:10.300 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd full ratio = 0.9 2024-04-22T06:30:10.300 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd nearfull ratio = 0.8 2024-04-22T06:30:10.300 INFO:teuthology.orchestra.run.smithi070.stdout: ms inject delay max = 1 2024-04-22T06:30:10.301 INFO:teuthology.orchestra.run.smithi070.stdout: ms inject delay probability = 0.005 2024-04-22T06:30:10.301 INFO:teuthology.orchestra.run.smithi070.stdout: ms inject delay type = client mds 2024-04-22T06:30:10.301 INFO:teuthology.orchestra.run.smithi070.stdout: ms inject socket failures = 2500 2024-04-22T06:30:10.301 INFO:teuthology.orchestra.run.smithi070.stdout: osd blocked scrub grace period = 3600 2024-04-22T06:30:10.301 INFO:teuthology.orchestra.run.smithi070.stdout: osd failsafe full ratio = 0.95 2024-04-22T06:30:10.301 INFO:teuthology.orchestra.run.smithi070.stdout: osd max markdown count = 1000 2024-04-22T06:30:10.301 INFO:teuthology.orchestra.run.smithi070.stdout: osd objectstore = bluestore 2024-04-22T06:30:10.301 INFO:teuthology.orchestra.run.smithi070.stdout: osd op complaint time = 180 2024-04-22T06:30:10.301 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:30:10.301 INFO:teuthology.orchestra.run.smithi070.stdout:[mgr] 2024-04-22T06:30:10.301 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2024-04-22T06:30:10.301 INFO:teuthology.orchestra.run.smithi070.stdout: debug mgr = 20 2024-04-22T06:30:10.301 INFO:teuthology.orchestra.run.smithi070.stdout: debug mon = 20 2024-04-22T06:30:10.301 INFO:teuthology.orchestra.run.smithi070.stdout: debug auth = 20 2024-04-22T06:30:10.302 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min pgs per osd = 4 2024-04-22T06:30:10.302 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min bytes per osd = 10 2024-04-22T06:30:10.302 INFO:teuthology.orchestra.run.smithi070.stdout: mgr/telemetry/nag = false 2024-04-22T06:30:10.302 INFO:teuthology.orchestra.run.smithi070.stdout: debug client = 20 2024-04-22T06:30:10.302 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:30:10.302 INFO:teuthology.orchestra.run.smithi070.stdout:[mon] 2024-04-22T06:30:10.302 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2024-04-22T06:30:10.302 INFO:teuthology.orchestra.run.smithi070.stdout: debug mon = 20 2024-04-22T06:30:10.302 INFO:teuthology.orchestra.run.smithi070.stdout: debug paxos = 20 2024-04-22T06:30:10.302 INFO:teuthology.orchestra.run.smithi070.stdout: debug auth = 20 2024-04-22T06:30:10.302 INFO:teuthology.orchestra.run.smithi070.stdout: mon data avail warn = 5 2024-04-22T06:30:10.302 INFO:teuthology.orchestra.run.smithi070.stdout: mon mgr mkfs grace = 240 2024-04-22T06:30:10.302 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min pgs per osd = 4 2024-04-22T06:30:10.302 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd reporter subtree level = osd 2024-04-22T06:30:10.303 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd prime pg temp = true 2024-04-22T06:30:10.303 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min bytes per osd = 10 2024-04-22T06:30:10.303 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:30:10.303 INFO:teuthology.orchestra.run.smithi070.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-22T06:30:10.303 INFO:teuthology.orchestra.run.smithi070.stdout: auth mon ticket ttl = 660 # 11m 2024-04-22T06:30:10.303 INFO:teuthology.orchestra.run.smithi070.stdout: auth service ticket ttl = 240 # 4m 2024-04-22T06:30:10.303 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:30:10.303 INFO:teuthology.orchestra.run.smithi070.stdout: # don't complain about insecure global_id in the test suite 2024-04-22T06:30:10.303 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-22T06:30:10.303 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-22T06:30:10.303 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:30:10.303 INFO:teuthology.orchestra.run.smithi070.stdout: # 1m isn't quite enough 2024-04-22T06:30:10.303 INFO:teuthology.orchestra.run.smithi070.stdout: mon_down_mkfs_grace = 2m 2024-04-22T06:30:10.304 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:30:10.304 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_filestore_osds = false 2024-04-22T06:30:10.304 INFO:teuthology.orchestra.run.smithi070.stdout: mon down mkfs grace = 300 2024-04-22T06:30:10.304 INFO:teuthology.orchestra.run.smithi070.stdout: mon op complaint time = 120 2024-04-22T06:30:10.304 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:30:10.304 INFO:teuthology.orchestra.run.smithi070.stdout:[client] 2024-04-22T06:30:10.304 INFO:teuthology.orchestra.run.smithi070.stdout: rgw cache enabled = true 2024-04-22T06:30:10.304 INFO:teuthology.orchestra.run.smithi070.stdout: rgw enable ops log = true 2024-04-22T06:30:10.304 INFO:teuthology.orchestra.run.smithi070.stdout: rgw enable usage log = true 2024-04-22T06:30:10.304 INFO:teuthology.orchestra.run.smithi070.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-22T06:30:10.304 INFO:teuthology.orchestra.run.smithi070.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-22T06:30:10.304 INFO:teuthology.orchestra.run.smithi070.stdout: client mount timeout = 600 2024-04-22T06:30:10.304 INFO:teuthology.orchestra.run.smithi070.stdout: client_shutdown_timeout = 120 2024-04-22T06:30:10.304 INFO:teuthology.orchestra.run.smithi070.stdout: debug client = 20 2024-04-22T06:30:10.305 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2024-04-22T06:30:10.305 INFO:teuthology.orchestra.run.smithi070.stdout: rados mon op timeout = 900 2024-04-22T06:30:10.305 INFO:teuthology.orchestra.run.smithi070.stdout: rados osd op timeout = 900 2024-04-22T06:30:10.305 INFO:teuthology.orchestra.run.smithi070.stdout:[mon.a] 2024-04-22T06:30:10.305 INFO:teuthology.orchestra.run.smithi070.stdout:[mon.b] 2024-04-22T06:30:10.305 INFO:teuthology.orchestra.run.smithi070.stdout:[mon.c] 2024-04-22T06:30:10.305 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.a] 2024-04-22T06:30:10.305 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.c] 2024-04-22T06:30:10.305 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.e] 2024-04-22T06:30:10.305 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.b] 2024-04-22T06:30:10.305 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.d] 2024-04-22T06:30:10.305 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.f] 2024-04-22T06:30:10.305 INFO:teuthology.orchestra.run.smithi070.stdout:[mds] 2024-04-22T06:30:10.305 INFO:teuthology.orchestra.run.smithi070.stdout: debug mds = 20 2024-04-22T06:30:10.305 INFO:teuthology.orchestra.run.smithi070.stdout: debug mds balancer = 20 2024-04-22T06:30:10.305 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2024-04-22T06:30:10.306 INFO:teuthology.orchestra.run.smithi070.stdout: mds bal fragment size max = 10000 2024-04-22T06:30:10.306 INFO:teuthology.orchestra.run.smithi070.stdout: mds bal merge size = 5 2024-04-22T06:30:10.306 INFO:teuthology.orchestra.run.smithi070.stdout: mds bal split bits = 3 2024-04-22T06:30:10.306 INFO:teuthology.orchestra.run.smithi070.stdout: mds bal split size = 100 2024-04-22T06:30:10.306 INFO:teuthology.orchestra.run.smithi070.stdout: mds debug frag = True 2024-04-22T06:30:10.306 INFO:teuthology.orchestra.run.smithi070.stdout: mds debug scatterstat = True 2024-04-22T06:30:10.306 INFO:teuthology.orchestra.run.smithi070.stdout: mds oft prefetch dirfrags = True 2024-04-22T06:30:10.306 INFO:teuthology.orchestra.run.smithi070.stdout: mds op complaint time = 180 2024-04-22T06:30:10.306 INFO:teuthology.orchestra.run.smithi070.stdout: mds verify scatter = True 2024-04-22T06:30:10.306 INFO:teuthology.orchestra.run.smithi070.stdout: mds_dir_prefetch = True 2024-04-22T06:30:10.306 INFO:teuthology.orchestra.run.smithi070.stdout: mds_sleep_rank_change = 5000000.0 2024-04-22T06:30:10.306 INFO:teuthology.orchestra.run.smithi070.stdout: mon client directed command retry = 5 2024-04-22T06:30:10.306 INFO:teuthology.orchestra.run.smithi070.stdout: ms inject delay max = 1 2024-04-22T06:30:10.306 INFO:teuthology.orchestra.run.smithi070.stdout: ms inject delay probability = 0.005 2024-04-22T06:30:10.307 INFO:teuthology.orchestra.run.smithi070.stdout: ms inject delay type = client mds osd 2024-04-22T06:30:10.307 INFO:teuthology.orchestra.run.smithi070.stdout: ms inject socket failures = 2500 2024-04-22T06:30:10.307 INFO:teuthology.orchestra.run.smithi070.stdout: osd op complaint time = 180 2024-04-22T06:30:10.307 INFO:teuthology.orchestra.run.smithi070.stdout: rados mon op timeout = 900 2024-04-22T06:30:10.307 INFO:teuthology.orchestra.run.smithi070.stdout: rados osd op timeout = 900 2024-04-22T06:30:10.307 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-22T06:30:10.307 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-22T06:30:10.390 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-22T06:30:10.391 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T06:30:10.391 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-22T06:30:10.406 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T06:30:10.406 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-22T06:30:10.462 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi044.front.sepia.ceph.com 2024-04-22T06:30:10.462 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T06:30:10.462 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-22T06:30:10.462 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-22T06:30:10.535 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T06:30:10.535 DEBUG:teuthology.orchestra.run.smithi044:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-22T06:30:10.594 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi070.front.sepia.ceph.com 2024-04-22T06:30:10.594 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T06:30:10.594 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-22T06:30:10.595 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-22T06:30:10.628 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T06:30:10.628 DEBUG:teuthology.orchestra.run.smithi070:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-22T06:30:10.683 INFO:tasks.ceph:Setting up mon nodes... 2024-04-22T06:30:10.683 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-22T06:30:10.684 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-22T06:30:10.733 INFO:teuthology.orchestra.run.smithi044.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-22T06:30:10.736 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-22T06:30:10.793 INFO:teuthology.orchestra.run.smithi070.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-22T06:30:10.799 INFO:tasks.ceph:Setting up mds nodes... 2024-04-22T06:30:10.799 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-22T06:30:10.849 INFO:teuthology.orchestra.run.smithi044.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-22T06:30:10.852 DEBUG:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-22T06:30:10.876 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-22T06:30:10.966 INFO:teuthology.orchestra.run.smithi044.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-22T06:30:10.968 DEBUG:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-22T06:30:10.992 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/mds/ceph-e && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.e /var/lib/ceph/mds/ceph-e/keyring 2024-04-22T06:30:11.082 INFO:teuthology.orchestra.run.smithi044.stdout:creating /var/lib/ceph/mds/ceph-e/keyring 2024-04-22T06:30:11.085 DEBUG:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-e 2024-04-22T06:30:11.109 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-22T06:30:11.158 INFO:teuthology.orchestra.run.smithi070.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-22T06:30:11.161 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-22T06:30:11.184 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-22T06:30:11.271 INFO:teuthology.orchestra.run.smithi070.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-22T06:30:11.274 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-22T06:30:11.296 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/mds/ceph-f && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.f /var/lib/ceph/mds/ceph-f/keyring 2024-04-22T06:30:11.385 INFO:teuthology.orchestra.run.smithi070.stdout:creating /var/lib/ceph/mds/ceph-f/keyring 2024-04-22T06:30:11.388 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-f 2024-04-22T06:30:11.415 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-22T06:30:11.415 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-22T06:30:11.456 INFO:teuthology.orchestra.run.smithi044.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-22T06:30:11.468 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-22T06:30:11.469 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi044.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi070.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-22T06:30:11.469 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-22T06:30:11.534 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-22T06:30:11.534 INFO:tasks.ceph:role: osd.0 2024-04-22T06:30:11.534 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi044.front.sepia.ceph.com 2024-04-22T06:30:11.534 DEBUG:teuthology.orchestra.run.smithi044:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-22T06:30:11.605 INFO:teuthology.orchestra.run.smithi044.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-22T06:30:11.605 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-22T06:30:11.605 INFO:teuthology.orchestra.run.smithi044.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-22T06:30:11.605 INFO:teuthology.orchestra.run.smithi044.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-22T06:30:11.605 INFO:teuthology.orchestra.run.smithi044.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-22T06:30:11.605 INFO:teuthology.orchestra.run.smithi044.stdout: = sunit=0 swidth=0 blks 2024-04-22T06:30:11.606 INFO:teuthology.orchestra.run.smithi044.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-22T06:30:11.606 INFO:teuthology.orchestra.run.smithi044.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-22T06:30:11.606 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-22T06:30:11.606 INFO:teuthology.orchestra.run.smithi044.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-22T06:30:12.106 INFO:teuthology.orchestra.run.smithi044.stdout:Discarding blocks...Done. 2024-04-22T06:30:12.108 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi044.front.sepia.ceph.com -o noatime 2024-04-22T06:30:12.108 DEBUG:teuthology.orchestra.run.smithi044:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-22T06:30:12.235 DEBUG:teuthology.orchestra.run.smithi044:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-22T06:30:12.267 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-22T06:30:12.333 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-22T06:30:12.333 INFO:tasks.ceph:role: osd.1 2024-04-22T06:30:12.333 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi044.front.sepia.ceph.com 2024-04-22T06:30:12.333 DEBUG:teuthology.orchestra.run.smithi044:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-22T06:30:12.406 INFO:teuthology.orchestra.run.smithi044.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-22T06:30:12.406 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-22T06:30:12.406 INFO:teuthology.orchestra.run.smithi044.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-22T06:30:12.406 INFO:teuthology.orchestra.run.smithi044.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-22T06:30:12.406 INFO:teuthology.orchestra.run.smithi044.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-22T06:30:12.406 INFO:teuthology.orchestra.run.smithi044.stdout: = sunit=0 swidth=0 blks 2024-04-22T06:30:12.406 INFO:teuthology.orchestra.run.smithi044.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-22T06:30:12.406 INFO:teuthology.orchestra.run.smithi044.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-22T06:30:12.406 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-22T06:30:12.406 INFO:teuthology.orchestra.run.smithi044.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-22T06:30:12.854 INFO:teuthology.orchestra.run.smithi044.stdout:Discarding blocks...Done. 2024-04-22T06:30:12.856 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi044.front.sepia.ceph.com -o noatime 2024-04-22T06:30:12.856 DEBUG:teuthology.orchestra.run.smithi044:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-22T06:30:12.945 DEBUG:teuthology.orchestra.run.smithi044:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-22T06:30:12.976 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-22T06:30:13.042 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-22T06:30:13.043 INFO:tasks.ceph:role: osd.2 2024-04-22T06:30:13.043 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi044.front.sepia.ceph.com 2024-04-22T06:30:13.043 DEBUG:teuthology.orchestra.run.smithi044:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-22T06:30:13.117 INFO:teuthology.orchestra.run.smithi044.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-22T06:30:13.117 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-22T06:30:13.117 INFO:teuthology.orchestra.run.smithi044.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-22T06:30:13.117 INFO:teuthology.orchestra.run.smithi044.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-22T06:30:13.117 INFO:teuthology.orchestra.run.smithi044.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-22T06:30:13.117 INFO:teuthology.orchestra.run.smithi044.stdout: = sunit=0 swidth=0 blks 2024-04-22T06:30:13.117 INFO:teuthology.orchestra.run.smithi044.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-22T06:30:13.118 INFO:teuthology.orchestra.run.smithi044.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-22T06:30:13.118 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-22T06:30:13.118 INFO:teuthology.orchestra.run.smithi044.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-22T06:30:13.595 INFO:teuthology.orchestra.run.smithi044.stdout:Discarding blocks...Done. 2024-04-22T06:30:13.598 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi044.front.sepia.ceph.com -o noatime 2024-04-22T06:30:13.598 DEBUG:teuthology.orchestra.run.smithi044:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-22T06:30:13.716 DEBUG:teuthology.orchestra.run.smithi044:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-22T06:30:13.746 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-22T06:30:13.810 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-22T06:30:13.811 INFO:tasks.ceph:role: osd.3 2024-04-22T06:30:13.811 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi044.front.sepia.ceph.com 2024-04-22T06:30:13.811 DEBUG:teuthology.orchestra.run.smithi044:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-22T06:30:13.883 INFO:teuthology.orchestra.run.smithi044.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-22T06:30:13.884 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-22T06:30:13.884 INFO:teuthology.orchestra.run.smithi044.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-22T06:30:13.884 INFO:teuthology.orchestra.run.smithi044.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-22T06:30:13.884 INFO:teuthology.orchestra.run.smithi044.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-22T06:30:13.884 INFO:teuthology.orchestra.run.smithi044.stdout: = sunit=0 swidth=0 blks 2024-04-22T06:30:13.884 INFO:teuthology.orchestra.run.smithi044.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-22T06:30:13.884 INFO:teuthology.orchestra.run.smithi044.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-22T06:30:13.884 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-22T06:30:13.884 INFO:teuthology.orchestra.run.smithi044.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-22T06:30:14.420 INFO:teuthology.orchestra.run.smithi044.stdout:Discarding blocks...Done. 2024-04-22T06:30:14.422 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi044.front.sepia.ceph.com -o noatime 2024-04-22T06:30:14.422 DEBUG:teuthology.orchestra.run.smithi044:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-22T06:30:14.562 DEBUG:teuthology.orchestra.run.smithi044:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-22T06:30:14.591 DEBUG:teuthology.orchestra.run.smithi044:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-22T06:30:14.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:14.690+0000 7f6ae5a0f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-22T06:30:14.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:14.690+0000 7f6ae5a0f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-22T06:30:14.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:14.690+0000 7f6ae5a0f640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-22T06:30:14.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:14.690+0000 7f6ae5a0f640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-22T06:30:14.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:14.690+0000 7f6ae5a0f640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-22T06:30:14.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:14.690+0000 7f6ae5a0f640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-22T06:30:17.541 DEBUG:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-22T06:30:17.571 DEBUG:teuthology.orchestra.run.smithi044:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-22T06:30:17.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:17.661+0000 7f2981a14640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-22T06:30:17.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:17.661+0000 7f2981a14640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-22T06:30:17.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:17.661+0000 7f2981a14640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-22T06:30:17.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:17.662+0000 7f2981a14640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-22T06:30:17.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:17.662+0000 7f2981a14640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-22T06:30:17.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:17.662+0000 7f2981a14640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-22T06:30:20.514 DEBUG:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-22T06:30:20.543 DEBUG:teuthology.orchestra.run.smithi044:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-22T06:30:20.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:20.633+0000 7fe582017640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-22T06:30:20.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:20.634+0000 7fe582017640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-22T06:30:20.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:20.634+0000 7fe582017640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-22T06:30:20.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:20.634+0000 7fe582017640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-22T06:30:20.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:20.634+0000 7fe582017640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-22T06:30:20.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:20.634+0000 7fe582017640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-22T06:30:23.501 DEBUG:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-22T06:30:23.530 DEBUG:teuthology.orchestra.run.smithi044:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-22T06:30:23.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:23.621+0000 7f3b1942c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-22T06:30:23.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:23.621+0000 7f3b1942c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-22T06:30:23.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:23.621+0000 7f3b1942c640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-22T06:30:23.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:23.622+0000 7f3b1942c640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-22T06:30:23.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:23.622+0000 7f3b1942c640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-22T06:30:23.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:23.622+0000 7f3b1942c640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-22T06:30:26.470 DEBUG:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-22T06:30:26.499 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-22T06:30:26.528 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-22T06:30:26.528 INFO:tasks.ceph:role: osd.4 2024-04-22T06:30:26.528 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi070.front.sepia.ceph.com 2024-04-22T06:30:26.528 DEBUG:teuthology.orchestra.run.smithi070:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-22T06:30:26.595 INFO:teuthology.orchestra.run.smithi070.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-22T06:30:26.595 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-22T06:30:26.595 INFO:teuthology.orchestra.run.smithi070.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-22T06:30:26.595 INFO:teuthology.orchestra.run.smithi070.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-22T06:30:26.595 INFO:teuthology.orchestra.run.smithi070.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-22T06:30:26.595 INFO:teuthology.orchestra.run.smithi070.stdout: = sunit=0 swidth=0 blks 2024-04-22T06:30:26.595 INFO:teuthology.orchestra.run.smithi070.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-22T06:30:26.595 INFO:teuthology.orchestra.run.smithi070.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-22T06:30:26.596 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-22T06:30:26.596 INFO:teuthology.orchestra.run.smithi070.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-22T06:30:26.627 INFO:teuthology.orchestra.run.smithi070.stdout:Discarding blocks...Done. 2024-04-22T06:30:26.630 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi070.front.sepia.ceph.com -o noatime 2024-04-22T06:30:26.630 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-22T06:30:26.705 DEBUG:teuthology.orchestra.run.smithi070:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-22T06:30:26.774 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-22T06:30:26.838 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-22T06:30:26.838 INFO:tasks.ceph:role: osd.5 2024-04-22T06:30:26.838 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi070.front.sepia.ceph.com 2024-04-22T06:30:26.838 DEBUG:teuthology.orchestra.run.smithi070:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-22T06:30:26.901 INFO:teuthology.orchestra.run.smithi070.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-22T06:30:26.901 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-22T06:30:26.901 INFO:teuthology.orchestra.run.smithi070.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-22T06:30:26.902 INFO:teuthology.orchestra.run.smithi070.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-22T06:30:26.902 INFO:teuthology.orchestra.run.smithi070.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-22T06:30:26.902 INFO:teuthology.orchestra.run.smithi070.stdout: = sunit=0 swidth=0 blks 2024-04-22T06:30:26.902 INFO:teuthology.orchestra.run.smithi070.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-22T06:30:26.902 INFO:teuthology.orchestra.run.smithi070.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-22T06:30:26.902 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-22T06:30:26.902 INFO:teuthology.orchestra.run.smithi070.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-22T06:30:26.934 INFO:teuthology.orchestra.run.smithi070.stdout:Discarding blocks...Done. 2024-04-22T06:30:26.936 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi070.front.sepia.ceph.com -o noatime 2024-04-22T06:30:26.936 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-22T06:30:27.003 DEBUG:teuthology.orchestra.run.smithi070:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-22T06:30:27.075 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-22T06:30:27.139 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-22T06:30:27.139 INFO:tasks.ceph:role: osd.6 2024-04-22T06:30:27.139 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi070.front.sepia.ceph.com 2024-04-22T06:30:27.139 DEBUG:teuthology.orchestra.run.smithi070:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-22T06:30:27.202 INFO:teuthology.orchestra.run.smithi070.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-22T06:30:27.202 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-22T06:30:27.202 INFO:teuthology.orchestra.run.smithi070.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-22T06:30:27.202 INFO:teuthology.orchestra.run.smithi070.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-22T06:30:27.202 INFO:teuthology.orchestra.run.smithi070.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-22T06:30:27.202 INFO:teuthology.orchestra.run.smithi070.stdout: = sunit=0 swidth=0 blks 2024-04-22T06:30:27.202 INFO:teuthology.orchestra.run.smithi070.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-22T06:30:27.202 INFO:teuthology.orchestra.run.smithi070.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-22T06:30:27.202 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-22T06:30:27.202 INFO:teuthology.orchestra.run.smithi070.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-22T06:30:27.234 INFO:teuthology.orchestra.run.smithi070.stdout:Discarding blocks...Done. 2024-04-22T06:30:27.236 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi070.front.sepia.ceph.com -o noatime 2024-04-22T06:30:27.236 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-22T06:30:27.313 DEBUG:teuthology.orchestra.run.smithi070:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-22T06:30:27.381 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-22T06:30:27.445 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-22T06:30:27.445 INFO:tasks.ceph:role: osd.7 2024-04-22T06:30:27.446 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi070.front.sepia.ceph.com 2024-04-22T06:30:27.446 DEBUG:teuthology.orchestra.run.smithi070:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-22T06:30:27.509 INFO:teuthology.orchestra.run.smithi070.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-22T06:30:27.509 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-22T06:30:27.509 INFO:teuthology.orchestra.run.smithi070.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-22T06:30:27.509 INFO:teuthology.orchestra.run.smithi070.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-22T06:30:27.509 INFO:teuthology.orchestra.run.smithi070.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-22T06:30:27.509 INFO:teuthology.orchestra.run.smithi070.stdout: = sunit=0 swidth=0 blks 2024-04-22T06:30:27.509 INFO:teuthology.orchestra.run.smithi070.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-22T06:30:27.509 INFO:teuthology.orchestra.run.smithi070.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-22T06:30:27.509 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-22T06:30:27.509 INFO:teuthology.orchestra.run.smithi070.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-22T06:30:27.541 INFO:teuthology.orchestra.run.smithi070.stdout:Discarding blocks...Done. 2024-04-22T06:30:27.543 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi070.front.sepia.ceph.com -o noatime 2024-04-22T06:30:27.543 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-22T06:30:27.610 DEBUG:teuthology.orchestra.run.smithi070:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-22T06:30:27.677 DEBUG:teuthology.orchestra.run.smithi070:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-22T06:30:27.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:27.774+0000 7f72b5387640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-22T06:30:27.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:27.774+0000 7f72b5387640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-22T06:30:27.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:27.774+0000 7f72b5387640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-22T06:30:27.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:27.774+0000 7f72b5387640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-22T06:30:27.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:27.774+0000 7f72b5387640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-22T06:30:27.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:27.775+0000 7f72b5387640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-22T06:30:30.610 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-22T06:30:30.637 DEBUG:teuthology.orchestra.run.smithi070:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-22T06:30:30.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:30.723+0000 7fcc8c82c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-22T06:30:30.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:30.723+0000 7fcc8c82c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-22T06:30:30.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:30.723+0000 7fcc8c82c640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-22T06:30:30.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:30.723+0000 7fcc8c82c640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-22T06:30:30.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:30.723+0000 7fcc8c82c640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-22T06:30:30.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:30.724+0000 7fcc8c82c640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-22T06:30:33.572 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-22T06:30:33.599 DEBUG:teuthology.orchestra.run.smithi070:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-22T06:30:33.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:33.688+0000 7f7037013640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-22T06:30:33.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:33.688+0000 7f7037013640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-22T06:30:33.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:33.688+0000 7f7037013640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-22T06:30:33.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:33.688+0000 7f7037013640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-22T06:30:33.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:33.688+0000 7f7037013640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-22T06:30:33.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:33.689+0000 7f7037013640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-22T06:30:36.533 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-22T06:30:36.561 DEBUG:teuthology.orchestra.run.smithi070:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-22T06:30:36.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:36.649+0000 7f737502c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-22T06:30:36.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:36.649+0000 7f737502c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-22T06:30:36.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:36.649+0000 7f737502c640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-22T06:30:36.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:36.649+0000 7f737502c640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-22T06:30:36.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:36.649+0000 7f737502c640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-22T06:30:36.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:36.650+0000 7f737502c640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-22T06:30:38.268 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T06:30:38.271 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T06:30:39.492 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-22T06:30:39.519 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-22T06:30:39.519 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T06:30:39.519 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-22T06:30:39.548 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T06:30:39.548 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-22T06:30:39.616 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T06:30:39.616 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-22T06:30:39.683 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T06:30:39.683 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/mds/ceph-e/keyring of=/dev/stdout 2024-04-22T06:30:39.749 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T06:30:39.749 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-22T06:30:39.815 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T06:30:39.815 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-22T06:30:39.881 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T06:30:39.881 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-22T06:30:39.947 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T06:30:39.947 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-22T06:30:40.012 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T06:30:40.012 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-22T06:30:40.041 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T06:30:40.041 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-22T06:30:40.108 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T06:30:40.108 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-22T06:30:40.175 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T06:30:40.175 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/mds/ceph-f/keyring of=/dev/stdout 2024-04-22T06:30:40.241 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T06:30:40.241 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-22T06:30:40.306 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T06:30:40.306 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-22T06:30:40.371 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T06:30:40.371 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-22T06:30:40.435 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T06:30:40.436 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-22T06:30:40.500 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T06:30:40.500 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-22T06:30:40.518 INFO:tasks.ceph:Adding keys to all mons... 2024-04-22T06:30:40.519 DEBUG:teuthology.orchestra.run.smithi044:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-22T06:30:40.561 DEBUG:teuthology.orchestra.run.smithi070:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-22T06:30:40.582 INFO:teuthology.orchestra.run.smithi070.stdout:[mgr.y] 2024-04-22T06:30:40.583 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQDyAyZmVmzVKxAAg35qJY8rMzlmQHosmCFbsg== 2024-04-22T06:30:40.583 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.a] 2024-04-22T06:30:40.583 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQDyAyZmphnJMhAARD2+mzQ4WsSByiUTnZWP5Q== 2024-04-22T06:30:40.583 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.c] 2024-04-22T06:30:40.583 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQDyAyZm0xG0ORAAUlQHvUd5nxCY0J5byj93fw== 2024-04-22T06:30:40.583 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.e] 2024-04-22T06:30:40.583 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQDzAyZmZ84QBRAAdQ25SWl5Mh34YMEFpTLqDg== 2024-04-22T06:30:40.583 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.0] 2024-04-22T06:30:40.583 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQD2AyZmeQYuKRAAg04o2N50LKpwkxAcXrQWJQ== 2024-04-22T06:30:40.583 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.1] 2024-04-22T06:30:40.583 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQD5AyZm+K59JxAASiveKAEppRJfZJvU41Og6g== 2024-04-22T06:30:40.583 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.2] 2024-04-22T06:30:40.583 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQD8AyZmjqrTJRAAcP4Nt9D84BWroDitxA3mDQ== 2024-04-22T06:30:40.583 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.3] 2024-04-22T06:30:40.583 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQD/AyZm9qkqJRAAK0ORQb+1jF6Ct+15Y4sztg== 2024-04-22T06:30:40.584 INFO:teuthology.orchestra.run.smithi070.stdout:[mgr.x] 2024-04-22T06:30:40.584 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQDyAyZm4ylkLxAAzYg7PVo6OeONhdpesB7dvA== 2024-04-22T06:30:40.584 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.b] 2024-04-22T06:30:40.584 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQDzAyZmLIaXCRAAC69W5a+/LxjDOmlc6JiXEg== 2024-04-22T06:30:40.584 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.d] 2024-04-22T06:30:40.584 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQDzAyZmmbNTEBAAWPuSZY4LtmcHLX5Kf95k5w== 2024-04-22T06:30:40.584 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.f] 2024-04-22T06:30:40.584 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQDzAyZmMf4gFxAAeDdTs4hLj+G8uJR7U88GCg== 2024-04-22T06:30:40.584 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.4] 2024-04-22T06:30:40.584 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQADBCZmdCdBLhAAvzpb9FkB1IpMrGGS2oXeUg== 2024-04-22T06:30:40.584 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.5] 2024-04-22T06:30:40.584 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQAGBCZm8Ys2KxAAFOY5blBfTPE829h1ItEnyg== 2024-04-22T06:30:40.584 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.6] 2024-04-22T06:30:40.584 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQAJBCZmvgEhKRAAQOw1XEGTDTSTGNSGs4PLzw== 2024-04-22T06:30:40.584 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.7] 2024-04-22T06:30:40.584 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQAMBCZmWyjOJhAARFzP6cG1lXbaYT/XaMgOUw== 2024-04-22T06:30:40.585 INFO:teuthology.orchestra.run.smithi070.stdout:[client.0] 2024-04-22T06:30:40.585 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQDzAyZmFFpZGxAA/TSDs+2/+GNLdk+vYGUYDA== 2024-04-22T06:30:40.586 INFO:teuthology.orchestra.run.smithi044.stdout:[mgr.y] 2024-04-22T06:30:40.586 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQDyAyZmVmzVKxAAg35qJY8rMzlmQHosmCFbsg== 2024-04-22T06:30:40.586 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.a] 2024-04-22T06:30:40.586 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQDyAyZmphnJMhAARD2+mzQ4WsSByiUTnZWP5Q== 2024-04-22T06:30:40.586 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.c] 2024-04-22T06:30:40.586 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQDyAyZm0xG0ORAAUlQHvUd5nxCY0J5byj93fw== 2024-04-22T06:30:40.586 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.e] 2024-04-22T06:30:40.586 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQDzAyZmZ84QBRAAdQ25SWl5Mh34YMEFpTLqDg== 2024-04-22T06:30:40.586 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.0] 2024-04-22T06:30:40.586 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQD2AyZmeQYuKRAAg04o2N50LKpwkxAcXrQWJQ== 2024-04-22T06:30:40.586 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.1] 2024-04-22T06:30:40.586 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQD5AyZm+K59JxAASiveKAEppRJfZJvU41Og6g== 2024-04-22T06:30:40.586 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.2] 2024-04-22T06:30:40.586 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQD8AyZmjqrTJRAAcP4Nt9D84BWroDitxA3mDQ== 2024-04-22T06:30:40.587 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.3] 2024-04-22T06:30:40.587 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQD/AyZm9qkqJRAAK0ORQb+1jF6Ct+15Y4sztg== 2024-04-22T06:30:40.587 INFO:teuthology.orchestra.run.smithi044.stdout:[mgr.x] 2024-04-22T06:30:40.587 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQDyAyZm4ylkLxAAzYg7PVo6OeONhdpesB7dvA== 2024-04-22T06:30:40.587 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.b] 2024-04-22T06:30:40.587 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQDzAyZmLIaXCRAAC69W5a+/LxjDOmlc6JiXEg== 2024-04-22T06:30:40.587 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.d] 2024-04-22T06:30:40.587 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQDzAyZmmbNTEBAAWPuSZY4LtmcHLX5Kf95k5w== 2024-04-22T06:30:40.587 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.f] 2024-04-22T06:30:40.587 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQDzAyZmMf4gFxAAeDdTs4hLj+G8uJR7U88GCg== 2024-04-22T06:30:40.587 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.4] 2024-04-22T06:30:40.587 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQADBCZmdCdBLhAAvzpb9FkB1IpMrGGS2oXeUg== 2024-04-22T06:30:40.587 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.5] 2024-04-22T06:30:40.587 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQAGBCZm8Ys2KxAAFOY5blBfTPE829h1ItEnyg== 2024-04-22T06:30:40.587 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.6] 2024-04-22T06:30:40.587 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQAJBCZmvgEhKRAAQOw1XEGTDTSTGNSGs4PLzw== 2024-04-22T06:30:40.588 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.7] 2024-04-22T06:30:40.588 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQAMBCZmWyjOJhAARFzP6cG1lXbaYT/XaMgOUw== 2024-04-22T06:30:40.588 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2024-04-22T06:30:40.588 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQDzAyZmFFpZGxAA/TSDs+2/+GNLdk+vYGUYDA== 2024-04-22T06:30:40.588 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-22T06:30:40.631 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-22T06:30:40.683 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-22T06:30:40.686 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-22T06:30:40.739 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-22T06:30:40.741 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-22T06:30:40.794 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-22T06:30:40.796 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-22T06:30:40.847 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T06:30:40.850 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T06:30:40.898 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T06:30:40.900 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T06:30:40.949 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T06:30:40.951 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T06:30:40.999 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T06:30:41.001 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T06:30:41.050 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-22T06:30:41.052 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-22T06:30:41.101 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-22T06:30:41.103 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-22T06:30:41.154 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-22T06:30:41.156 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-22T06:30:41.208 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-22T06:30:41.210 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-22T06:30:41.262 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T06:30:41.264 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T06:30:41.312 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T06:30:41.314 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T06:30:41.363 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T06:30:41.365 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T06:30:41.413 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T06:30:41.415 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T06:30:41.464 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-22T06:30:41.466 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-22T06:30:41.517 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-22T06:30:41.517 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-22T06:30:41.541 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-22T06:30:41.636 DEBUG:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-22T06:30:41.660 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-22T06:30:41.683 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-22T06:30:41.778 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-22T06:30:41.801 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-22T06:30:41.864 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-22T06:30:41.957 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-22T06:30:41.980 DEBUG:teuthology.orchestra.run.smithi044:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-22T06:30:41.982 DEBUG:teuthology.orchestra.run.smithi070:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-22T06:30:42.035 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-22T06:30:42.035 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-22T06:30:42.035 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-22T06:30:42.038 INFO:tasks.ceph.mon.a:Started 2024-04-22T06:30:42.038 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-22T06:30:42.038 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-22T06:30:42.077 INFO:tasks.ceph.mon.b:Started 2024-04-22T06:30:42.077 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-22T06:30:42.077 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-22T06:30:42.079 INFO:tasks.ceph.mon.c:Started 2024-04-22T06:30:42.079 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-22T06:30:42.080 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-22T06:30:42.080 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-22T06:30:42.082 INFO:tasks.ceph.mgr.y:Started 2024-04-22T06:30:42.082 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-22T06:30:42.082 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-22T06:30:42.084 INFO:tasks.ceph.mgr.x:Started 2024-04-22T06:30:42.084 INFO:tasks.ceph:Setting crush tunables to default 2024-04-22T06:30:42.084 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph osd crush tunables default 2024-04-22T06:30:42.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.186+0000 7f5588304640 1 Processor -- start 2024-04-22T06:30:42.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.186+0000 7f5588304640 1 -- start start 2024-04-22T06:30:42.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.187+0000 7f5588304640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f558006f330 0x7f55800ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:42.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.187+0000 7f5588304640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f558006f710 0x7f5580152d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:42.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.187+0000 7f5588304640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55800c3e00 con 0x7f55800cd1b0 2024-04-22T06:30:42.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.187+0000 7f5588304640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f55800c3bb0 con 0x7f558006f710 2024-04-22T06:30:42.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.187+0000 7f5588304640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f55800c4a10 con 0x7f558006f330 2024-04-22T06:30:42.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.187+0000 7f558589f640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f558006f710 0x7f5580152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:42.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.187+0000 7f558589f640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f558006f710 0x7f5580152d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57410/0 (socket says 172.21.15.44:57410) 2024-04-22T06:30:42.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.187+0000 7f55868a1640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f55800cd1b0 0x7f5580159390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:55264/0 (socket says 172.21.15.44:55264) 2024-04-22T06:30:42.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.187+0000 7f558589f640 1 -- 172.21.15.44:0/3887520390 learned_addr learned my addr 172.21.15.44:0/3887520390 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:30:42.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.187+0000 7f55860a0640 1 --2- 172.21.15.44:0/3887520390 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f558006f330 0x7f55800ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:42.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.187+0000 7f558589f640 1 -- 172.21.15.44:0/3887520390 >> v2:172.21.15.44:3300/0 conn(0x7f558006f710 msgr2=0x7f5580152d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-22T06:30:42.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.187+0000 7f558589f640 1 -- 172.21.15.44:0/3887520390 >> v2:172.21.15.44:3300/0 conn(0x7f558006f710 msgr2=0x7f5580152d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T06:30:42.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.187+0000 7f558589f640 1 --2- 172.21.15.44:0/3887520390 >> v2:172.21.15.44:3300/0 conn(0x7f558006f710 0x7f5580152d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T06:30:42.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.187+0000 7f558589f640 1 --2- 172.21.15.44:0/3887520390 >> v2:172.21.15.44:3300/0 conn(0x7f558006f710 0x7f5580152d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:30:42.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.187+0000 7f55860a0640 1 -- 172.21.15.44:0/3887520390 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f558006f330 msgr2=0x7f55800ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-22T06:30:42.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.187+0000 7f55860a0640 1 -- 172.21.15.44:0/3887520390 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f558006f330 msgr2=0x7f55800ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T06:30:42.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.187+0000 7f55860a0640 1 --2- 172.21.15.44:0/3887520390 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f558006f330 0x7f55800ccc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T06:30:42.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.187+0000 7f55860a0640 1 --2- 172.21.15.44:0/3887520390 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f558006f330 0x7f55800ccc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:30:42.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.342+0000 7f558509e640 1 -- 172.21.15.44:0/3887520390 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 464760171 0 0) 0x7f55800c3e00 con 0x7f55800cd1b0 2024-04-22T06:30:42.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.342+0000 7f558509e640 1 -- 172.21.15.44:0/3887520390 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f556c003520 con 0x7f55800cd1b0 2024-04-22T06:30:42.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.342+0000 7f558509e640 1 -- 172.21.15.44:0/3887520390 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1235417721 0 0) 0x7f556c003520 con 0x7f55800cd1b0 2024-04-22T06:30:42.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.342+0000 7f558509e640 1 -- 172.21.15.44:0/3887520390 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f558006f330 msgr2=0x7f55800ccc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:30:42.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.342+0000 7f558509e640 1 --2- 172.21.15.44:0/3887520390 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f558006f330 0x7f55800ccc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:42.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.342+0000 7f558509e640 1 -- 172.21.15.44:0/3887520390 >> v2:172.21.15.44:3300/0 conn(0x7f558006f710 msgr2=0x7f5580152d70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:30:42.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.342+0000 7f558509e640 1 --2- 172.21.15.44:0/3887520390 >> v2:172.21.15.44:3300/0 conn(0x7f558006f710 0x7f5580152d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:42.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.342+0000 7f558509e640 1 -- 172.21.15.44:0/3887520390 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55800c40e0 con 0x7f55800cd1b0 2024-04-22T06:30:42.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.342+0000 7f558509e640 1 -- 172.21.15.44:0/3887520390 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1237339257 0 0) 0x7f55780019f0 con 0x7f55800cd1b0 2024-04-22T06:30:42.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.342+0000 7f558509e640 1 -- 172.21.15.44:0/3887520390 >> v1:172.21.15.44:6789/0 conn(0x7f55800cd1b0 legacy=0x7f5580159390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:42.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.342+0000 7f558509e640 1 --2- 172.21.15.44:0/3887520390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f556c0039b0 0x7f556c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:42.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.342+0000 7f558509e640 1 --2- 172.21.15.44:0/3887520390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f556c004800 0x7f556c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:42.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.342+0000 7f558509e640 1 --2- 172.21.15.44:0/3887520390 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f558006f330 0x7f556c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:42.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.342+0000 7f558509e640 1 -- 172.21.15.44:0/3887520390 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f55800c3bb0 con 0x7f556c004800 2024-04-22T06:30:42.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.342+0000 7f558509e640 1 -- 172.21.15.44:0/3887520390 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f55800c4a10 con 0x7f556c0039b0 2024-04-22T06:30:42.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.342+0000 7f558509e640 1 -- 172.21.15.44:0/3887520390 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f556c00ba70 con 0x7f558006f330 2024-04-22T06:30:42.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.342+0000 7f558589f640 1 --2- 172.21.15.44:0/3887520390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f556c004800 0x7f556c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:42.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.343+0000 7f55868a1640 1 --2- 172.21.15.44:0/3887520390 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f558006f330 0x7f556c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:42.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.343+0000 7f55860a0640 1 --2- 172.21.15.44:0/3887520390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f556c0039b0 0x7f556c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:42.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.343+0000 7f558589f640 1 -- 172.21.15.44:0/3887520390 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f558006f330 msgr2=0x7f556c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:30:42.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.343+0000 7f558589f640 1 --2- 172.21.15.44:0/3887520390 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f558006f330 0x7f556c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:42.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.343+0000 7f558589f640 1 -- 172.21.15.44:0/3887520390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f556c0039b0 msgr2=0x7f556c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:30:42.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.343+0000 7f558589f640 1 --2- 172.21.15.44:0/3887520390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f556c0039b0 0x7f556c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:42.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.343+0000 7f558589f640 1 -- 172.21.15.44:0/3887520390 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f556c003520 con 0x7f556c004800 2024-04-22T06:30:42.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.344+0000 7f558589f640 1 --2- 172.21.15.44:0/3887520390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f556c004800 0x7f556c004c80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5570009100 tx=0x7f556c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8efb69be7bcb6ac6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:30:42.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.344+0000 7f558509e640 1 -- 172.21.15.44:0/3887520390 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5570034070 con 0x7f556c004800 2024-04-22T06:30:42.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.344+0000 7f558509e640 1 -- 172.21.15.44:0/3887520390 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5570030630 con 0x7f556c004800 2024-04-22T06:30:42.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.344+0000 7f558509e640 1 -- 172.21.15.44:0/3887520390 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557000f040 con 0x7f556c004800 2024-04-22T06:30:42.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.344+0000 7f5588304640 1 -- 172.21.15.44:0/3887520390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f556c004800 msgr2=0x7f556c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:42.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.344+0000 7f5588304640 1 --2- 172.21.15.44:0/3887520390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f556c004800 0x7f556c004c80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5570009100 tx=0x7f556c005830 comp rx=0 tx=0).stop 2024-04-22T06:30:42.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.344+0000 7f55868a1640 1 -- 172.21.15.44:0/3887520390 reap_dead start 2024-04-22T06:30:42.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.344+0000 7f5588304640 1 -- 172.21.15.44:0/3887520390 shutdown_connections 2024-04-22T06:30:42.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.344+0000 7f5588304640 1 -- 172.21.15.44:0/3887520390 >> 172.21.15.44:0/3887520390 conn(0x7f558006cf00 msgr2=0x7f558006d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:30:42.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.345+0000 7f5588304640 1 -- 172.21.15.44:0/3887520390 shutdown_connections 2024-04-22T06:30:42.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.345+0000 7f5588304640 1 -- 172.21.15.44:0/3887520390 wait complete. 2024-04-22T06:30:42.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.346+0000 7f5588304640 1 Processor -- start 2024-04-22T06:30:42.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.346+0000 7f5588304640 1 -- start start 2024-04-22T06:30:42.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.346+0000 7f5588304640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f556c004800 0x7f55801481a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:42.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.346+0000 7f5588304640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55800c9480 0x7f5580149ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:42.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.347+0000 7f5588304640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f558014cc50 0x7f5580146140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:42.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.347+0000 7f5588304640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f55800cc760 con 0x7f558014cc50 2024-04-22T06:30:42.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.347+0000 7f5588304640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f55800c46a0 con 0x7f55800c9480 2024-04-22T06:30:42.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.347+0000 7f5588304640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f55800c6930 con 0x7f556c004800 2024-04-22T06:30:42.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.347+0000 7f55868a1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f558014cc50 0x7f5580146140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:42.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.347+0000 7f55868a1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f558014cc50 0x7f5580146140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57414/0 (socket says 172.21.15.44:57414) 2024-04-22T06:30:42.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.347+0000 7f55868a1640 1 -- 172.21.15.44:0/1628753314 learned_addr learned my addr 172.21.15.44:0/1628753314 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:30:42.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.347+0000 7f55860a0640 1 --2- 172.21.15.44:0/1628753314 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f556c004800 0x7f55801481a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:42.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.347+0000 7f558589f640 1 --2- 172.21.15.44:0/1628753314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55800c9480 0x7f5580149ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:42.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.347+0000 7f55868a1640 1 -- 172.21.15.44:0/1628753314 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f556c004800 msgr2=0x7f55801481a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:42.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.347+0000 7f55868a1640 1 --2- 172.21.15.44:0/1628753314 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f556c004800 0x7f55801481a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:42.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.347+0000 7f55868a1640 1 -- 172.21.15.44:0/1628753314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55800c9480 msgr2=0x7f5580149ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:42.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.347+0000 7f55868a1640 1 --2- 172.21.15.44:0/1628753314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55800c9480 0x7f5580149ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:42.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.347+0000 7f55868a1640 1 -- 172.21.15.44:0/1628753314 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5580071ce0 con 0x7f558014cc50 2024-04-22T06:30:42.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.347+0000 7f55868a1640 1 --2- 172.21.15.44:0/1628753314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f558014cc50 0x7f5580146140 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f558006ced0 tx=0x7f55780079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:30:42.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.347+0000 7f5576ffd640 1 -- 172.21.15.44:0/1628753314 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557801a070 con 0x7f558014cc50 2024-04-22T06:30:42.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.347+0000 7f5576ffd640 1 -- 172.21.15.44:0/1628753314 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f557800f8b0 con 0x7f558014cc50 2024-04-22T06:30:42.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.347+0000 7f5588304640 1 -- 172.21.15.44:0/1628753314 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55800722a0 con 0x7f558014cc50 2024-04-22T06:30:42.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.347+0000 7f5576ffd640 1 -- 172.21.15.44:0/1628753314 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557800fca0 con 0x7f558014cc50 2024-04-22T06:30:42.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.347+0000 7f5588304640 1 -- 172.21.15.44:0/1628753314 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5580072c20 con 0x7f558014cc50 2024-04-22T06:30:42.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.347+0000 7f5576ffd640 1 -- 172.21.15.44:0/1628753314 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f557801d070 con 0x7f558014cc50 2024-04-22T06:30:42.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.347+0000 7f5576ffd640 1 -- 172.21.15.44:0/1628753314 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f557801f7d0 con 0x7f558014cc50 2024-04-22T06:30:42.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.348+0000 7f5588304640 1 -- 172.21.15.44:0/1628753314 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5580072a10 con 0x7f558014cc50 2024-04-22T06:30:42.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.350+0000 7f5576ffd640 1 -- 172.21.15.44:0/1628753314 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f5578014450 con 0x7f558014cc50 2024-04-22T06:30:42.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.418+0000 7f5588304640 1 -- 172.21.15.44:0/1628753314 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f5580068cc0 con 0x7f558014cc50 2024-04-22T06:30:42.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.420+0000 7f5576ffd640 1 -- 172.21.15.44:0/1628753314 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f557801fa20 con 0x7f558014cc50 2024-04-22T06:30:42.419 INFO:teuthology.orchestra.run.smithi044.stderr:adjusted tunables profile to default 2024-04-22T06:30:42.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.421+0000 7f5588304640 1 -- 172.21.15.44:0/1628753314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f558014cc50 msgr2=0x7f5580146140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:42.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.421+0000 7f5588304640 1 --2- 172.21.15.44:0/1628753314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f558014cc50 0x7f5580146140 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f558006ced0 tx=0x7f55780079d0 comp rx=0 tx=0).stop 2024-04-22T06:30:42.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.421+0000 7f5588304640 1 -- 172.21.15.44:0/1628753314 shutdown_connections 2024-04-22T06:30:42.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.421+0000 7f5588304640 1 --2- 172.21.15.44:0/1628753314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f558014cc50 0x7f5580146140 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:42.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.421+0000 7f5588304640 1 --2- 172.21.15.44:0/1628753314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55800c9480 0x7f5580149ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:42.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.421+0000 7f5588304640 1 --2- 172.21.15.44:0/1628753314 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f556c004800 0x7f55801481a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:42.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.421+0000 7f5588304640 1 -- 172.21.15.44:0/1628753314 >> 172.21.15.44:0/1628753314 conn(0x7f558006cf00 msgr2=0x7f558014d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:30:42.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.421+0000 7f5588304640 1 -- 172.21.15.44:0/1628753314 shutdown_connections 2024-04-22T06:30:42.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:42.421+0000 7f5588304640 1 -- 172.21.15.44:0/1628753314 wait complete. 2024-04-22T06:30:42.431 INFO:tasks.ceph:check_enable_crimson: False 2024-04-22T06:30:42.431 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-22T06:30:42.431 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T06:30:42.431 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-22T06:30:42.455 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T06:30:42.455 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-22T06:30:42.464 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T06:30:42.465+0000 7fc89d821040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-22T06:30:42.475 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T06:30:42.476+0000 7fba14012040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-22T06:30:42.498 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T06:30:42.499 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-22T06:30:42.562 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T06:30:42.563 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-22T06:30:42.605 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T06:30:42.606+0000 7fc89d821040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-22T06:30:42.626 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T06:30:42.626 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-22T06:30:42.650 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T06:30:42.650 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-22T06:30:42.658 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T06:30:42.660+0000 7fba14012040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-22T06:30:42.660 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T06:30:42.661+0000 7fc89d821040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-22T06:30:42.682 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T06:30:42.682 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-22T06:30:42.715 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T06:30:42.716+0000 7fba14012040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-22T06:30:42.745 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T06:30:42.745 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-22T06:30:42.809 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd new 2b4a6692-53c9-472d-b2f3-1dd29a6679ed 0 2024-04-22T06:30:42.918 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T06:30:42.919+0000 7fc89d821040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-22T06:30:42.945 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.946+0000 7fd0aeb06640 1 Processor -- start 2024-04-22T06:30:42.945 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.946+0000 7fd0aeb06640 1 -- start start 2024-04-22T06:30:42.947 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.947+0000 7fd0aeb06640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd0a8075440 0x7fd0a80c5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:42.947 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.947+0000 7fd0aeb06640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd0a80cd6b0 0x7fd0a80cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:42.947 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.947+0000 7fd0aeb06640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0a80c6f10 con 0x7fd0a8075820 2024-04-22T06:30:42.947 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.947+0000 7fd0aeb06640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fd0a80ce000 con 0x7fd0a8075440 2024-04-22T06:30:42.947 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.947+0000 7fd0aeb06640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fd0a80c7180 con 0x7fd0a80cd6b0 2024-04-22T06:30:42.947 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.947+0000 7fd0ad0a3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd0a80cd6b0 0x7fd0a80cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:42.947 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.947+0000 7fd0ad0a3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd0a80cd6b0 0x7fd0a80cdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41400/0 (socket says 172.21.15.70:41400) 2024-04-22T06:30:42.948 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.947+0000 7fd0ad0a3640 1 -- 172.21.15.70:0/1010626737 learned_addr learned my addr 172.21.15.70:0/1010626737 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-22T06:30:42.948 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.947+0000 7fd0ac8a2640 1 --2- 172.21.15.70:0/1010626737 >> v2:172.21.15.44:3300/0 conn(0x7fd0a8075440 0x7fd0a80c5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:42.948 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.947+0000 7fd0ad0a3640 1 -- 172.21.15.70:0/1010626737 >> v2:172.21.15.44:3300/0 conn(0x7fd0a8075440 msgr2=0x7fd0a80c5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:30:42.948 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.947+0000 7fd0ad0a3640 1 --2- 172.21.15.70:0/1010626737 >> v2:172.21.15.44:3300/0 conn(0x7fd0a8075440 0x7fd0a80c5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:42.948 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.947+0000 7fd0ad0a3640 1 -- 172.21.15.70:0/1010626737 >> v1:172.21.15.44:6789/0 conn(0x7fd0a8075820 legacy=0x7fd0a8152a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:42.948 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.947+0000 7fd0ad0a3640 1 -- 172.21.15.70:0/1010626737 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a80c7460 con 0x7fd0a80cd6b0 2024-04-22T06:30:42.948 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.947+0000 7fd0ac8a2640 1 --2- 172.21.15.70:0/1010626737 >> v2:172.21.15.44:3300/0 conn(0x7fd0a8075440 0x7fd0a80c5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:30:42.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.949+0000 7fd0ad0a3640 1 --2- 172.21.15.70:0/1010626737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0a80cd6b0 0x7fd0a80cdac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd0a80686f0 tx=0x7fd0a002bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6538999f3925cba server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:30:42.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.949+0000 7fd0a77fe640 1 -- 172.21.15.70:0/1010626737 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a002cb20 con 0x7fd0a80cd6b0 2024-04-22T06:30:42.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.949+0000 7fd0a77fe640 1 -- 172.21.15.70:0/1010626737 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0a002ccc0 con 0x7fd0a80cd6b0 2024-04-22T06:30:42.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.949+0000 7fd0a77fe640 1 -- 172.21.15.70:0/1010626737 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0034610 con 0x7fd0a80cd6b0 2024-04-22T06:30:42.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.949+0000 7fd0aeb06640 1 -- 172.21.15.70:0/1010626737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0a80cd6b0 msgr2=0x7fd0a80cdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:42.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.949+0000 7fd0aeb06640 1 --2- 172.21.15.70:0/1010626737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0a80cd6b0 0x7fd0a80cdac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd0a80686f0 tx=0x7fd0a002bf80 comp rx=0 tx=0).stop 2024-04-22T06:30:42.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.949+0000 7fd0aeb06640 1 -- 172.21.15.70:0/1010626737 shutdown_connections 2024-04-22T06:30:42.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.949+0000 7fd0aeb06640 1 --2- 172.21.15.70:0/1010626737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0a80cd6b0 0x7fd0a80cdac0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:42.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.949+0000 7fd0aeb06640 1 --2- 172.21.15.70:0/1010626737 >> v2:172.21.15.44:3300/0 conn(0x7fd0a8075440 0x7fd0a80c5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:42.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.949+0000 7fd0aeb06640 1 -- 172.21.15.70:0/1010626737 >> 172.21.15.70:0/1010626737 conn(0x7fd0a806fcc0 msgr2=0x7fd0a80700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:30:42.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.949+0000 7fd0aeb06640 1 -- 172.21.15.70:0/1010626737 shutdown_connections 2024-04-22T06:30:42.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.949+0000 7fd0aeb06640 1 -- 172.21.15.70:0/1010626737 wait complete. 2024-04-22T06:30:42.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.950+0000 7fd0aeb06640 1 Processor -- start 2024-04-22T06:30:42.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.951+0000 7fd0aeb06640 1 -- start start 2024-04-22T06:30:42.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.951+0000 7fd0aeb06640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0a8075440 0x7fd0a80ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:42.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.951+0000 7fd0aeb06640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0a8075820 0x7fd0a80cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:42.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.951+0000 7fd0aeb06640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0a80cd6b0 0x7fd0a8156cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:42.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.951+0000 7fd0aeb06640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a8153170 con 0x7fd0a8075820 2024-04-22T06:30:42.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.951+0000 7fd0aeb06640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a80c7a20 con 0x7fd0a8075440 2024-04-22T06:30:42.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.951+0000 7fd0aeb06640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd0a81532f0 con 0x7fd0a80cd6b0 2024-04-22T06:30:42.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.951+0000 7fd0ac8a2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0a8075440 0x7fd0a80ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:42.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.951+0000 7fd0ac8a2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0a8075440 0x7fd0a80ccdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41410/0 (socket says 172.21.15.70:41410) 2024-04-22T06:30:42.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.951+0000 7fd0ac8a2640 1 -- 172.21.15.70:0/1543159137 learned_addr learned my addr 172.21.15.70:0/1543159137 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-22T06:30:42.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.951+0000 7fd0ad0a3640 1 --2- 172.21.15.70:0/1543159137 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0a80cd6b0 0x7fd0a8156cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:42.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.952+0000 7fd0a7fff640 1 --2- 172.21.15.70:0/1543159137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0a8075820 0x7fd0a80cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:42.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.952+0000 7fd0ac8a2640 1 -- 172.21.15.70:0/1543159137 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0a80cd6b0 msgr2=0x7fd0a8156cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:42.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.952+0000 7fd0ac8a2640 1 --2- 172.21.15.70:0/1543159137 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0a80cd6b0 0x7fd0a8156cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:42.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.952+0000 7fd0ac8a2640 1 -- 172.21.15.70:0/1543159137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0a8075820 msgr2=0x7fd0a80cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:42.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.952+0000 7fd0ac8a2640 1 --2- 172.21.15.70:0/1543159137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0a8075820 0x7fd0a80cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:42.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.952+0000 7fd0ac8a2640 1 -- 172.21.15.70:0/1543159137 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a80cb680 con 0x7fd0a8075440 2024-04-22T06:30:42.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.952+0000 7fd0ad0a3640 1 --2- 172.21.15.70:0/1543159137 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0a80cd6b0 0x7fd0a8156cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:30:42.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.952+0000 7fd0ac8a2640 1 --2- 172.21.15.70:0/1543159137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0a8075440 0x7fd0a80ccdc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00ebe0 tx=0x7fd09c00c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:30:42.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.952+0000 7fd0a57fa640 1 -- 172.21.15.70:0/1543159137 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c017070 con 0x7fd0a8075440 2024-04-22T06:30:42.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.952+0000 7fd0a57fa640 1 -- 172.21.15.70:0/1543159137 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd09c00cc30 con 0x7fd0a8075440 2024-04-22T06:30:42.955 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.952+0000 7fd0aeb06640 1 -- 172.21.15.70:0/1543159137 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a80cb920 con 0x7fd0a8075440 2024-04-22T06:30:42.955 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.952+0000 7fd0a57fa640 1 -- 172.21.15.70:0/1543159137 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c013470 con 0x7fd0a8075440 2024-04-22T06:30:42.955 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.952+0000 7fd0aeb06640 1 -- 172.21.15.70:0/1543159137 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a80cbee0 con 0x7fd0a8075440 2024-04-22T06:30:42.955 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.952+0000 7fd0a57fa640 1 -- 172.21.15.70:0/1543159137 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd09c01a070 con 0x7fd0a8075440 2024-04-22T06:30:42.955 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.952+0000 7fd0a57fa640 1 -- 172.21.15.70:0/1543159137 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fd09c013df0 con 0x7fd0a8075440 2024-04-22T06:30:42.955 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.953+0000 7fd0aeb06640 1 -- 172.21.15.70:0/1543159137 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0a8069310 con 0x7fd0a8075440 2024-04-22T06:30:42.955 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:42.955+0000 7fd0a57fa640 1 -- 172.21.15.70:0/1543159137 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fd09c013610 con 0x7fd0a8075440 2024-04-22T06:30:42.966 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T06:30:42.967+0000 7fc89d821040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-22T06:30:42.976 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T06:30:42.978+0000 7fba14012040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-22T06:30:43.024 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T06:30:43.026+0000 7fba14012040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-22T06:30:43.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.026+0000 7fd0aeb06640 1 -- 172.21.15.70:0/1543159137 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2b4a6692-53c9-472d-b2f3-1dd29a6679ed", "id": 0} v 0) -- 0x7fd0a8151b30 con 0x7fd0a8075440 2024-04-22T06:30:43.029 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.029+0000 7fd0a57fa640 1 -- 172.21.15.70:0/1543159137 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2b4a6692-53c9-472d-b2f3-1dd29a6679ed", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fd09c004020 con 0x7fd0a8075440 2024-04-22T06:30:43.029 INFO:teuthology.orchestra.run.smithi070.stdout:0 2024-04-22T06:30:43.030 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.030+0000 7fd0aeb06640 1 -- 172.21.15.70:0/1543159137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0a8075440 msgr2=0x7fd0a80ccdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:43.030 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.030+0000 7fd0aeb06640 1 --2- 172.21.15.70:0/1543159137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0a8075440 0x7fd0a80ccdc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00ebe0 tx=0x7fd09c00c520 comp rx=0 tx=0).stop 2024-04-22T06:30:43.030 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.031+0000 7fd0aeb06640 1 -- 172.21.15.70:0/1543159137 shutdown_connections 2024-04-22T06:30:43.030 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.031+0000 7fd0aeb06640 1 --2- 172.21.15.70:0/1543159137 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0a80cd6b0 0x7fd0a8156cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:43.030 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.031+0000 7fd0aeb06640 1 --2- 172.21.15.70:0/1543159137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0a8075820 0x7fd0a80cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:43.030 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.031+0000 7fd0aeb06640 1 --2- 172.21.15.70:0/1543159137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0a8075440 0x7fd0a80ccdc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:43.031 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.031+0000 7fd0aeb06640 1 -- 172.21.15.70:0/1543159137 >> 172.21.15.70:0/1543159137 conn(0x7fd0a806fcc0 msgr2=0x7fd0a80ca620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:30:43.031 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.031+0000 7fd0aeb06640 1 -- 172.21.15.70:0/1543159137 shutdown_connections 2024-04-22T06:30:43.031 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.031+0000 7fd0aeb06640 1 -- 172.21.15.70:0/1543159137 wait complete. 2024-04-22T06:30:43.040 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd new 70a6370a-7c0a-4d64-a0e4-b2e726b4f291 1 2024-04-22T06:30:43.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.136+0000 7f4eb5463640 1 Processor -- start 2024-04-22T06:30:43.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.137+0000 7f4eb5463640 1 -- start start 2024-04-22T06:30:43.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.137+0000 7f4eb5463640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f4eb01551e0 0x7f4eb00ce790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:43.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.137+0000 7f4eb5463640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4eb01555c0 0x7f4eb00cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:43.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.137+0000 7f4eb5463640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4eb015ca70 0x7f4eb015ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:43.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.137+0000 7f4eb5463640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f4eb015b060 con 0x7f4eb01551e0 2024-04-22T06:30:43.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.137+0000 7f4eb5463640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f4eb015b300 con 0x7f4eb015ca70 2024-04-22T06:30:43.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.137+0000 7f4eb5463640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f4eb006c460 con 0x7f4eb01555c0 2024-04-22T06:30:43.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.137+0000 7f4eaf7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4eb015ca70 0x7f4eb015ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:43.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.137+0000 7f4eae7fc640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4eb01555c0 0x7f4eb00cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:43.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.137+0000 7f4eaf7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4eb015ca70 0x7f4eb015ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41424/0 (socket says 172.21.15.70:41424) 2024-04-22T06:30:43.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.137+0000 7f4eae7fc640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4eb01555c0 0x7f4eb00cecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:48918/0 (socket says 172.21.15.70:48918) 2024-04-22T06:30:43.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.138+0000 7f4eaf7fe640 1 -- 172.21.15.70:0/111832055 learned_addr learned my addr 172.21.15.70:0/111832055 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-22T06:30:43.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.138+0000 7f4eaeffd640 1 --2- 172.21.15.70:0/111832055 >> v2:172.21.15.44:3300/0 conn(0x7f4eb01551e0 0x7f4eb00ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:43.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.138+0000 7f4eaf7fe640 1 -- 172.21.15.70:0/111832055 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4eb01555c0 msgr2=0x7f4eb00cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:30:43.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.138+0000 7f4eaf7fe640 1 --2- 172.21.15.70:0/111832055 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4eb01555c0 0x7f4eb00cecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:43.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.138+0000 7f4eaf7fe640 1 -- 172.21.15.70:0/111832055 >> v2:172.21.15.44:3300/0 conn(0x7f4eb01551e0 msgr2=0x7f4eb00ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:30:43.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.138+0000 7f4eaf7fe640 1 --2- 172.21.15.70:0/111832055 >> v2:172.21.15.44:3300/0 conn(0x7f4eb01551e0 0x7f4eb00ce790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:43.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.138+0000 7f4eaf7fe640 1 -- 172.21.15.70:0/111832055 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb015b5c0 con 0x7f4eb015ca70 2024-04-22T06:30:43.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.138+0000 7f4eae7fc640 1 --2- 172.21.15.70:0/111832055 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4eb01555c0 0x7f4eb00cecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:30:43.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.138+0000 7f4eaf7fe640 1 --2- 172.21.15.70:0/111832055 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4eb015ca70 0x7f4eb015ce80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4eb00cf740 tx=0x7f4ea802bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c29c9f4e66ce4ea8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:30:43.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.139+0000 7f4eadffb640 1 -- 172.21.15.70:0/111832055 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea802cb00 con 0x7f4eb015ca70 2024-04-22T06:30:43.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.139+0000 7f4eadffb640 1 -- 172.21.15.70:0/111832055 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ea802cca0 con 0x7f4eb015ca70 2024-04-22T06:30:43.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.139+0000 7f4eadffb640 1 -- 172.21.15.70:0/111832055 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8034610 con 0x7f4eb015ca70 2024-04-22T06:30:43.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.139+0000 7f4eb5463640 1 -- 172.21.15.70:0/111832055 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4eb015ca70 msgr2=0x7f4eb015ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:43.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.139+0000 7f4eb5463640 1 --2- 172.21.15.70:0/111832055 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4eb015ca70 0x7f4eb015ce80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4eb00cf740 tx=0x7f4ea802bf60 comp rx=0 tx=0).stop 2024-04-22T06:30:43.141 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.139+0000 7f4eb5463640 1 -- 172.21.15.70:0/111832055 shutdown_connections 2024-04-22T06:30:43.141 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.139+0000 7f4eb5463640 1 --2- 172.21.15.70:0/111832055 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4eb015ca70 0x7f4eb015ce80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:43.141 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.139+0000 7f4eb5463640 1 --2- 172.21.15.70:0/111832055 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4eb01555c0 0x7f4eb00cecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:43.141 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.139+0000 7f4eb5463640 1 --2- 172.21.15.70:0/111832055 >> v2:172.21.15.44:3300/0 conn(0x7f4eb01551e0 0x7f4eb00ce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:43.141 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.139+0000 7f4eb5463640 1 -- 172.21.15.70:0/111832055 >> 172.21.15.70:0/111832055 conn(0x7f4eb006fad0 msgr2=0x7f4eb0074ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:30:43.141 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.139+0000 7f4eb5463640 1 -- 172.21.15.70:0/111832055 shutdown_connections 2024-04-22T06:30:43.141 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.139+0000 7f4eb5463640 1 -- 172.21.15.70:0/111832055 wait complete. 2024-04-22T06:30:43.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.141+0000 7f4eb5463640 1 Processor -- start 2024-04-22T06:30:43.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.141+0000 7f4eb5463640 1 -- start start 2024-04-22T06:30:43.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.141+0000 7f4eb5463640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4eb01551e0 0x7f4eb0153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:43.144 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.141+0000 7f4eb5463640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4eb015ca70 0x7f4eb01f2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:43.144 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.141+0000 7f4eb5463640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4eb01f3cf0 0x7f4eb01f28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:43.144 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.141+0000 7f4eb5463640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb015be60 con 0x7f4eb015ca70 2024-04-22T06:30:43.144 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.141+0000 7f4eb5463640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb015bb80 con 0x7f4eb01551e0 2024-04-22T06:30:43.144 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.141+0000 7f4eb5463640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f4eb015c550 con 0x7f4eb01f3cf0 2024-04-22T06:30:43.144 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.141+0000 7f4eaeffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4eb01551e0 0x7f4eb0153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:43.144 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.142+0000 7f4eaf7fe640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4eb01f3cf0 0x7f4eb01f28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:43.144 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.142+0000 7f4eaeffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4eb01551e0 0x7f4eb0153730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41426/0 (socket says 172.21.15.70:41426) 2024-04-22T06:30:43.144 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.142+0000 7f4eaeffd640 1 -- 172.21.15.70:0/797861400 learned_addr learned my addr 172.21.15.70:0/797861400 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-22T06:30:43.144 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.142+0000 7f4eae7fc640 1 --2- 172.21.15.70:0/797861400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4eb015ca70 0x7f4eb01f2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:43.144 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.142+0000 7f4eaeffd640 1 -- 172.21.15.70:0/797861400 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4eb01f3cf0 msgr2=0x7f4eb01f28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:43.144 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.142+0000 7f4eaeffd640 1 --2- 172.21.15.70:0/797861400 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4eb01f3cf0 0x7f4eb01f28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:43.144 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.142+0000 7f4eaeffd640 1 -- 172.21.15.70:0/797861400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4eb015ca70 msgr2=0x7f4eb01f2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:43.144 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.142+0000 7f4eaeffd640 1 --2- 172.21.15.70:0/797861400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4eb015ca70 0x7f4eb01f2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:43.145 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.142+0000 7f4eaeffd640 1 -- 172.21.15.70:0/797861400 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb00c6be0 con 0x7f4eb01551e0 2024-04-22T06:30:43.145 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.142+0000 7f4eaeffd640 1 --2- 172.21.15.70:0/797861400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4eb01551e0 0x7f4eb0153730 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4ea000e8c0 tx=0x7f4ea000ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:30:43.145 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.142+0000 7f4e977fe640 1 -- 172.21.15.70:0/797861400 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0017070 con 0x7f4eb01551e0 2024-04-22T06:30:43.145 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.142+0000 7f4e977fe640 1 -- 172.21.15.70:0/797861400 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ea000c940 con 0x7f4eb01551e0 2024-04-22T06:30:43.145 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.142+0000 7f4e977fe640 1 -- 172.21.15.70:0/797861400 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea000cd10 con 0x7f4eb01551e0 2024-04-22T06:30:43.145 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.142+0000 7f4eb5463640 1 -- 172.21.15.70:0/797861400 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eb01f41e0 con 0x7f4eb01551e0 2024-04-22T06:30:43.145 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.142+0000 7f4eb5463640 1 -- 172.21.15.70:0/797861400 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eb01f47a0 con 0x7f4eb01551e0 2024-04-22T06:30:43.145 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.142+0000 7f4e977fe640 1 -- 172.21.15.70:0/797861400 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4ea001a070 con 0x7f4eb01551e0 2024-04-22T06:30:43.145 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.142+0000 7f4e977fe640 1 -- 172.21.15.70:0/797861400 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f4ea0013470 con 0x7f4eb01551e0 2024-04-22T06:30:43.145 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.143+0000 7f4eb5463640 1 -- 172.21.15.70:0/797861400 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eb0068cc0 con 0x7f4eb01551e0 2024-04-22T06:30:43.145 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.145+0000 7f4e977fe640 1 -- 172.21.15.70:0/797861400 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f4ea0006020 con 0x7f4eb01551e0 2024-04-22T06:30:43.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.217+0000 7f4eb5463640 1 -- 172.21.15.70:0/797861400 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd new", "uuid": "70a6370a-7c0a-4d64-a0e4-b2e726b4f291", "id": 1} v 0) -- 0x7f4eb00ca0f0 con 0x7f4eb01551e0 2024-04-22T06:30:43.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.222+0000 7f4e977fe640 1 -- 172.21.15.70:0/797861400 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "70a6370a-7c0a-4d64-a0e4-b2e726b4f291", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f4ea0005990 con 0x7f4eb01551e0 2024-04-22T06:30:43.221 INFO:teuthology.orchestra.run.smithi070.stdout:1 2024-04-22T06:30:43.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.223+0000 7f4eb5463640 1 -- 172.21.15.70:0/797861400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4eb01551e0 msgr2=0x7f4eb0153730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:43.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.223+0000 7f4eb5463640 1 --2- 172.21.15.70:0/797861400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4eb01551e0 0x7f4eb0153730 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4ea000e8c0 tx=0x7f4ea000ed80 comp rx=0 tx=0).stop 2024-04-22T06:30:43.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.223+0000 7f4eb5463640 1 -- 172.21.15.70:0/797861400 shutdown_connections 2024-04-22T06:30:43.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.223+0000 7f4eb5463640 1 --2- 172.21.15.70:0/797861400 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4eb01f3cf0 0x7f4eb01f28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:43.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.223+0000 7f4eb5463640 1 --2- 172.21.15.70:0/797861400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4eb015ca70 0x7f4eb01f2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:43.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.223+0000 7f4eb5463640 1 --2- 172.21.15.70:0/797861400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4eb01551e0 0x7f4eb0153730 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:43.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.223+0000 7f4eb5463640 1 -- 172.21.15.70:0/797861400 >> 172.21.15.70:0/797861400 conn(0x7f4eb006fad0 msgr2=0x7f4eb0071130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:30:43.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.223+0000 7f4eb5463640 1 -- 172.21.15.70:0/797861400 shutdown_connections 2024-04-22T06:30:43.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.223+0000 7f4eb5463640 1 -- 172.21.15.70:0/797861400 wait complete. 2024-04-22T06:30:43.233 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd new f2e51a62-ca91-470d-bd5a-5710c61deee4 2 2024-04-22T06:30:43.330 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.330+0000 7fb740984640 1 Processor -- start 2024-04-22T06:30:43.330 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.330+0000 7fb740984640 1 -- start start 2024-04-22T06:30:43.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.331+0000 7fb740984640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb73c071aa0 0x7fb73c067990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:43.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.331+0000 7fb740984640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb73c073970 con 0x7fb73c06e320 2024-04-22T06:30:43.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.331+0000 7fb740984640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb73c06ebb0 con 0x7fb73c06e700 2024-04-22T06:30:43.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.331+0000 7fb740984640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fb73c158f10 con 0x7fb73c071aa0 2024-04-22T06:30:43.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.331+0000 7fb73ad9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb73c06e700 0x7fb73c162960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36902/0 (socket says 172.21.15.70:36902) 2024-04-22T06:30:43.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.331+0000 7fb73ad9d640 1 -- 172.21.15.70:0/2490231142 learned_addr learned my addr 172.21.15.70:0/2490231142 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-04-22T06:30:43.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.331+0000 7fb739d9b640 1 --2- 172.21.15.70:0/2490231142 >> v2:172.21.15.44:3300/0 conn(0x7fb73c071aa0 0x7fb73c067990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:43.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.331+0000 7fb73959a640 1 -- 172.21.15.70:0/2490231142 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 974891093 0 0) 0x7fb73c06ebb0 con 0x7fb73c06e700 2024-04-22T06:30:43.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.331+0000 7fb73959a640 1 -- 172.21.15.70:0/2490231142 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb724003520 con 0x7fb73c06e700 2024-04-22T06:30:43.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.332+0000 7fb73959a640 1 -- 172.21.15.70:0/2490231142 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2656973294 0 0) 0x7fb73c073970 con 0x7fb73c06e320 2024-04-22T06:30:43.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.332+0000 7fb73959a640 1 -- 172.21.15.70:0/2490231142 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb73c06ebb0 con 0x7fb73c06e320 2024-04-22T06:30:43.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.332+0000 7fb739d9b640 1 -- 172.21.15.70:0/2490231142 >> v1:172.21.15.70:6789/0 conn(0x7fb73c06e700 legacy=0x7fb73c162960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:43.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.332+0000 7fb73959a640 1 -- 172.21.15.70:0/2490231142 <== mon.1 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2302519234 0 0) 0x7fb724003520 con 0x7fb73c06e700 2024-04-22T06:30:43.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.332+0000 7fb739d9b640 1 -- 172.21.15.70:0/2490231142 >> v1:172.21.15.44:6789/0 conn(0x7fb73c06e320 legacy=0x7fb73c073260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:43.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.332+0000 7fb739d9b640 1 -- 172.21.15.70:0/2490231142 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb73c0c91b0 con 0x7fb73c071aa0 2024-04-22T06:30:43.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.333+0000 7fb739d9b640 1 --2- 172.21.15.70:0/2490231142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb73c071aa0 0x7fb73c067990 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb730009720 tx=0x7fb73002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7393b1bdfcd854a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:30:43.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.333+0000 7fb73959a640 1 -- 172.21.15.70:0/2490231142 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7340019f0 con 0x7fb73c071aa0 2024-04-22T06:30:43.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.333+0000 7fb73959a640 1 -- 172.21.15.70:0/2490231142 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb730037070 con 0x7fb73c071aa0 2024-04-22T06:30:43.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.333+0000 7fb73959a640 1 -- 172.21.15.70:0/2490231142 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7300317c0 con 0x7fb73c071aa0 2024-04-22T06:30:43.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.334+0000 7fb740984640 1 -- 172.21.15.70:0/2490231142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb73c071aa0 msgr2=0x7fb73c067990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:43.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.334+0000 7fb740984640 1 --2- 172.21.15.70:0/2490231142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb73c071aa0 0x7fb73c067990 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb730009720 tx=0x7fb73002b860 comp rx=0 tx=0).stop 2024-04-22T06:30:43.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.334+0000 7fb740984640 1 -- 172.21.15.70:0/2490231142 shutdown_connections 2024-04-22T06:30:43.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.334+0000 7fb740984640 1 --2- 172.21.15.70:0/2490231142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb73c071aa0 0x7fb73c067990 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:43.340 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.334+0000 7fb740984640 1 -- 172.21.15.70:0/2490231142 >> 172.21.15.70:0/2490231142 conn(0x7fb73c0ccff0 msgr2=0x7fb73c0cd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:30:43.340 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.334+0000 7fb740984640 1 -- 172.21.15.70:0/2490231142 shutdown_connections 2024-04-22T06:30:43.340 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.334+0000 7fb740984640 1 -- 172.21.15.70:0/2490231142 wait complete. 2024-04-22T06:30:43.340 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.335+0000 7fb740984640 1 Processor -- start 2024-04-22T06:30:43.340 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.335+0000 7fb740984640 1 -- start start 2024-04-22T06:30:43.340 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.336+0000 7fb740984640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb73c06e320 0x7fb73c156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:43.340 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.336+0000 7fb740984640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb73c06e700 0x7fb73c0c2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:43.340 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.336+0000 7fb740984640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb73c071aa0 0x7fb73c1f2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:43.340 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.336+0000 7fb740984640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb73c072d40 con 0x7fb73c06e320 2024-04-22T06:30:43.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.336+0000 7fb740984640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb73c072a60 con 0x7fb73c071aa0 2024-04-22T06:30:43.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.336+0000 7fb740984640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb73c068750 con 0x7fb73c06e700 2024-04-22T06:30:43.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.336+0000 7fb739d9b640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb73c06e700 0x7fb73c0c2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:43.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.336+0000 7fb73ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb73c071aa0 0x7fb73c1f2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:43.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.336+0000 7fb73a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb73c06e320 0x7fb73c156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:43.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.336+0000 7fb739d9b640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb73c06e700 0x7fb73c0c2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:48938/0 (socket says 172.21.15.70:48938) 2024-04-22T06:30:43.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.336+0000 7fb73ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb73c071aa0 0x7fb73c1f2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41434/0 (socket says 172.21.15.70:41434) 2024-04-22T06:30:43.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.336+0000 7fb739d9b640 1 -- 172.21.15.70:0/2414200210 learned_addr learned my addr 172.21.15.70:0/2414200210 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-22T06:30:43.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.336+0000 7fb73a59c640 1 -- 172.21.15.70:0/2414200210 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb73c06e700 msgr2=0x7fb73c0c2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:43.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.336+0000 7fb73a59c640 1 --2- 172.21.15.70:0/2414200210 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb73c06e700 0x7fb73c0c2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:43.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.336+0000 7fb73a59c640 1 -- 172.21.15.70:0/2414200210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb73c071aa0 msgr2=0x7fb73c1f2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:43.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.336+0000 7fb73a59c640 1 --2- 172.21.15.70:0/2414200210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb73c071aa0 0x7fb73c1f2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:43.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.336+0000 7fb73a59c640 1 -- 172.21.15.70:0/2414200210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb73c0cc500 con 0x7fb73c06e320 2024-04-22T06:30:43.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.337+0000 7fb73a59c640 1 --2- 172.21.15.70:0/2414200210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb73c06e320 0x7fb73c156f10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb72c00c390 tx=0x7fb72c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:30:43.342 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.337+0000 7fb72affd640 1 -- 172.21.15.70:0/2414200210 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72c017070 con 0x7fb73c06e320 2024-04-22T06:30:43.342 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.337+0000 7fb72affd640 1 -- 172.21.15.70:0/2414200210 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb72c0052d0 con 0x7fb73c06e320 2024-04-22T06:30:43.342 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.337+0000 7fb740984640 1 -- 172.21.15.70:0/2414200210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb73c151a80 con 0x7fb73c06e320 2024-04-22T06:30:43.342 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.337+0000 7fb72affd640 1 -- 172.21.15.70:0/2414200210 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72c003830 con 0x7fb73c06e320 2024-04-22T06:30:43.342 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.337+0000 7fb740984640 1 -- 172.21.15.70:0/2414200210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb73c0cbeb0 con 0x7fb73c06e320 2024-04-22T06:30:43.342 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.337+0000 7fb72affd640 1 -- 172.21.15.70:0/2414200210 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb72c01a070 con 0x7fb73c06e320 2024-04-22T06:30:43.342 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.337+0000 7fb72affd640 1 -- 172.21.15.70:0/2414200210 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fb72c012420 con 0x7fb73c06e320 2024-04-22T06:30:43.342 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.338+0000 7fb740984640 1 -- 172.21.15.70:0/2414200210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb73c069310 con 0x7fb73c06e320 2024-04-22T06:30:43.343 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.342+0000 7fb72affd640 1 -- 172.21.15.70:0/2414200210 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fb72c021070 con 0x7fb73c06e320 2024-04-22T06:30:43.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.415+0000 7fb740984640 1 -- 172.21.15.70:0/2414200210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f2e51a62-ca91-470d-bd5a-5710c61deee4", "id": 2} v 0) -- 0x7fb73c067ed0 con 0x7fb73c06e320 2024-04-22T06:30:43.418 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.419+0000 7fb72affd640 1 -- 172.21.15.70:0/2414200210 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f2e51a62-ca91-470d-bd5a-5710c61deee4", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fb72c011650 con 0x7fb73c06e320 2024-04-22T06:30:43.418 INFO:teuthology.orchestra.run.smithi070.stdout:2 2024-04-22T06:30:43.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.420+0000 7fb740984640 1 -- 172.21.15.70:0/2414200210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb73c06e320 msgr2=0x7fb73c156f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:43.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.420+0000 7fb740984640 1 --2- 172.21.15.70:0/2414200210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb73c06e320 0x7fb73c156f10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb72c00c390 tx=0x7fb72c001e70 comp rx=0 tx=0).stop 2024-04-22T06:30:43.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.420+0000 7fb740984640 1 -- 172.21.15.70:0/2414200210 shutdown_connections 2024-04-22T06:30:43.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.420+0000 7fb740984640 1 --2- 172.21.15.70:0/2414200210 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb73c06e700 0x7fb73c0c2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:43.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.420+0000 7fb740984640 1 --2- 172.21.15.70:0/2414200210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb73c071aa0 0x7fb73c1f2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:43.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.420+0000 7fb740984640 1 --2- 172.21.15.70:0/2414200210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb73c06e320 0x7fb73c156f10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:43.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.420+0000 7fb740984640 1 -- 172.21.15.70:0/2414200210 >> 172.21.15.70:0/2414200210 conn(0x7fb73c0ccff0 msgr2=0x7fb73c073260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:30:43.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.420+0000 7fb740984640 1 -- 172.21.15.70:0/2414200210 shutdown_connections 2024-04-22T06:30:43.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.420+0000 7fb740984640 1 -- 172.21.15.70:0/2414200210 wait complete. 2024-04-22T06:30:43.430 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd new bc514203-9af9-4f1f-b18e-765bdd5de6e7 3 2024-04-22T06:30:43.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.531+0000 7fb129fe6640 1 Processor -- start 2024-04-22T06:30:43.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.531+0000 7fb129fe6640 1 -- start start 2024-04-22T06:30:43.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.532+0000 7fb129fe6640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb1240cea30 0x7fb1240c66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:43.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.532+0000 7fb129fe6640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb124071ad0 con 0x7fb1240cee10 2024-04-22T06:30:43.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.532+0000 7fb129fe6640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1240c8030 con 0x7fb12414f490 2024-04-22T06:30:43.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.532+0000 7fb129fe6640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb1240c8c40 con 0x7fb1240cea30 2024-04-22T06:30:43.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.532+0000 7fb1237fe640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb1240cea30 0x7fb1240c66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:43.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.532+0000 7fb123fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb12414f490 0x7fb12414f900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36908/0 (socket says 172.21.15.70:36908) 2024-04-22T06:30:43.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.532+0000 7fb123fff640 1 -- 172.21.15.70:0/1335851340 learned_addr learned my addr 172.21.15.70:0/1335851340 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-04-22T06:30:43.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.532+0000 7fb1237fe640 1 -- 172.21.15.70:0/1335851340 >> v1:172.21.15.70:6789/0 conn(0x7fb12414f490 legacy=0x7fb12414f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:43.533 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.533+0000 7fb1237fe640 1 -- 172.21.15.70:0/1335851340 >> v1:172.21.15.44:6789/0 conn(0x7fb1240cee10 legacy=0x7fb1240711b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:30:43.533 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.533+0000 7fb1237fe640 1 -- 172.21.15.70:0/1335851340 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1240c8310 con 0x7fb1240cea30 2024-04-22T06:30:43.533 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.532+0000 7fb122ffd640 1 -- 172.21.15.70:0/1335851340 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 815114904 0 0) 0x7fb1240c8030 con 0x7fb12414f490 2024-04-22T06:30:43.534 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.533+0000 7fb1237fe640 1 --2- 172.21.15.70:0/1335851340 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb1240cea30 0x7fb1240c66e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb1140097d0 tx=0x7fb11402b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=760182526221a4a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:30:43.534 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.533+0000 7fb122ffd640 1 -- 172.21.15.70:0/1335851340 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb114035020 con 0x7fb1240cea30 2024-04-22T06:30:43.534 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.533+0000 7fb122ffd640 1 -- 172.21.15.70:0/1335851340 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb114031890 con 0x7fb1240cea30 2024-04-22T06:30:43.534 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.533+0000 7fb122ffd640 1 -- 172.21.15.70:0/1335851340 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb114031c60 con 0x7fb1240cea30 2024-04-22T06:30:43.534 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.534+0000 7fb129fe6640 1 -- 172.21.15.70:0/1335851340 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb1240cea30 msgr2=0x7fb1240c66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:43.534 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.534+0000 7fb129fe6640 1 --2- 172.21.15.70:0/1335851340 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb1240cea30 0x7fb1240c66e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb1140097d0 tx=0x7fb11402b930 comp rx=0 tx=0).stop 2024-04-22T06:30:43.534 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.534+0000 7fb129fe6640 1 -- 172.21.15.70:0/1335851340 shutdown_connections 2024-04-22T06:30:43.535 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.534+0000 7fb129fe6640 1 --2- 172.21.15.70:0/1335851340 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb1240cea30 0x7fb1240c66e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:43.535 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.534+0000 7fb129fe6640 1 -- 172.21.15.70:0/1335851340 >> 172.21.15.70:0/1335851340 conn(0x7fb1240737f0 msgr2=0x7fb12406f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:30:43.535 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.534+0000 7fb129fe6640 1 -- 172.21.15.70:0/1335851340 shutdown_connections 2024-04-22T06:30:43.535 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.534+0000 7fb129fe6640 1 -- 172.21.15.70:0/1335851340 wait complete. 2024-04-22T06:30:43.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.535+0000 7fb129fe6640 1 Processor -- start 2024-04-22T06:30:43.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.535+0000 7fb129fe6640 1 -- start start 2024-04-22T06:30:43.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.536+0000 7fb129fe6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1240cea30 0x7fb124141eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:43.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.536+0000 7fb129fe6640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb1240cee10 0x7fb124157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:43.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.536+0000 7fb129fe6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb12414f490 0x7fb12406bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:43.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.536+0000 7fb129fe6640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb124153590 con 0x7fb1240cea30 2024-04-22T06:30:43.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.536+0000 7fb129fe6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb1240c88d0 con 0x7fb12414f490 2024-04-22T06:30:43.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.536+0000 7fb129fe6640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb124159230 con 0x7fb1240cee10 2024-04-22T06:30:43.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.536+0000 7fb11bfff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb1240cee10 0x7fb124157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:43.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.536+0000 7fb123fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb12414f490 0x7fb12406bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:43.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.536+0000 7fb11bfff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb1240cee10 0x7fb124157110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:48942/0 (socket says 172.21.15.70:48942) 2024-04-22T06:30:43.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.536+0000 7fb11bfff640 1 -- 172.21.15.70:0/3552273476 learned_addr learned my addr 172.21.15.70:0/3552273476 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-22T06:30:43.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.536+0000 7fb1237fe640 1 --2- 172.21.15.70:0/3552273476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1240cea30 0x7fb124141eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:43.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.536+0000 7fb11bfff640 1 -- 172.21.15.70:0/3552273476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb12414f490 msgr2=0x7fb12406bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:43.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.536+0000 7fb11bfff640 1 --2- 172.21.15.70:0/3552273476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb12414f490 0x7fb12406bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:43.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.536+0000 7fb11bfff640 1 -- 172.21.15.70:0/3552273476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1240cea30 msgr2=0x7fb124141eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:43.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.536+0000 7fb11bfff640 1 --2- 172.21.15.70:0/3552273476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1240cea30 0x7fb124141eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:43.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.536+0000 7fb11bfff640 1 -- 172.21.15.70:0/3552273476 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb124158db0 con 0x7fb1240cee10 2024-04-22T06:30:43.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.537+0000 7fb11bfff640 1 --2- 172.21.15.70:0/3552273476 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb1240cee10 0x7fb124157110 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb10c00c430 tx=0x7fb10c00c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:30:43.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.537+0000 7fb120ff9640 1 -- 172.21.15.70:0/3552273476 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10c011790 con 0x7fb1240cee10 2024-04-22T06:30:43.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.537+0000 7fb120ff9640 1 -- 172.21.15.70:0/3552273476 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb10c017070 con 0x7fb1240cee10 2024-04-22T06:30:43.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.537+0000 7fb129fe6640 1 -- 172.21.15.70:0/3552273476 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb12406d370 con 0x7fb1240cee10 2024-04-22T06:30:43.539 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.537+0000 7fb120ff9640 1 -- 172.21.15.70:0/3552273476 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10c011bb0 con 0x7fb1240cee10 2024-04-22T06:30:43.539 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.537+0000 7fb129fe6640 1 -- 172.21.15.70:0/3552273476 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb124142a50 con 0x7fb1240cee10 2024-04-22T06:30:43.539 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.537+0000 7fb120ff9640 1 -- 172.21.15.70:0/3552273476 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb10c01c460 con 0x7fb1240cee10 2024-04-22T06:30:43.539 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.537+0000 7fb120ff9640 1 -- 172.21.15.70:0/3552273476 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fb10c012c20 con 0x7fb1240cee10 2024-04-22T06:30:43.539 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.538+0000 7fb129fe6640 1 -- 172.21.15.70:0/3552273476 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb124068cc0 con 0x7fb1240cee10 2024-04-22T06:30:43.539 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.540+0000 7fb120ff9640 1 -- 172.21.15.70:0/3552273476 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fb10c011d50 con 0x7fb1240cee10 2024-04-22T06:30:43.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.633+0000 7fb129fe6640 1 -- 172.21.15.70:0/3552273476 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd new", "uuid": "bc514203-9af9-4f1f-b18e-765bdd5de6e7", "id": 3} v 0) -- 0x7fb124156bd0 con 0x7fb1240cee10 2024-04-22T06:30:43.637 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.638+0000 7fb120ff9640 1 -- 172.21.15.70:0/3552273476 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bc514203-9af9-4f1f-b18e-765bdd5de6e7", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fb10c012790 con 0x7fb1240cee10 2024-04-22T06:30:43.637 INFO:teuthology.orchestra.run.smithi070.stdout:3 2024-04-22T06:30:43.638 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.639+0000 7fb129fe6640 1 -- 172.21.15.70:0/3552273476 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb1240cee10 msgr2=0x7fb124157110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:43.638 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.639+0000 7fb129fe6640 1 --2- 172.21.15.70:0/3552273476 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb1240cee10 0x7fb124157110 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb10c00c430 tx=0x7fb10c00c8f0 comp rx=0 tx=0).stop 2024-04-22T06:30:43.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.639+0000 7fb129fe6640 1 -- 172.21.15.70:0/3552273476 shutdown_connections 2024-04-22T06:30:43.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.639+0000 7fb129fe6640 1 --2- 172.21.15.70:0/3552273476 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb1240cee10 0x7fb124157110 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:43.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.639+0000 7fb129fe6640 1 --2- 172.21.15.70:0/3552273476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb12414f490 0x7fb12406bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:43.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.639+0000 7fb129fe6640 1 --2- 172.21.15.70:0/3552273476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1240cea30 0x7fb124141eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:43.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.639+0000 7fb129fe6640 1 -- 172.21.15.70:0/3552273476 >> 172.21.15.70:0/3552273476 conn(0x7fb1240737f0 msgr2=0x7fb1240c5bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:30:43.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.639+0000 7fb129fe6640 1 -- 172.21.15.70:0/3552273476 shutdown_connections 2024-04-22T06:30:43.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.639+0000 7fb129fe6640 1 -- 172.21.15.70:0/3552273476 wait complete. 2024-04-22T06:30:43.649 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd new 357c439a-6e08-4c19-ac18-b68e6c890497 4 2024-04-22T06:30:43.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.746+0000 7f7ae8a43640 1 Processor -- start 2024-04-22T06:30:43.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.746+0000 7f7ae8a43640 1 -- start start 2024-04-22T06:30:43.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.747+0000 7f7ae8a43640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ae4144340 0x7f7ae40756c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:43.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.747+0000 7f7ae8a43640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f7ae40c8430 0x7f7ae415ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:43.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.747+0000 7f7ae8a43640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7ae4144720 0x7f7ae415f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:43.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.747+0000 7f7ae8a43640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f7ae40723e0 con 0x7f7ae40c8430 2024-04-22T06:30:43.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.747+0000 7f7ae8a43640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f7ae40726a0 con 0x7f7ae4144720 2024-04-22T06:30:43.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.747+0000 7f7ae8a43640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7ae4072980 con 0x7f7ae4144340 2024-04-22T06:30:43.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.747+0000 7f7ae259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ae4144340 0x7f7ae40756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:43.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.747+0000 7f7ae2d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7ae4144720 0x7f7ae415f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:43.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.747+0000 7f7ae259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ae4144340 0x7f7ae40756c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:48952/0 (socket says 172.21.15.70:48952) 2024-04-22T06:30:43.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.747+0000 7f7ae259c640 1 -- 172.21.15.70:0/1768933461 learned_addr learned my addr 172.21.15.70:0/1768933461 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-22T06:30:43.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.747+0000 7f7ae1d9b640 1 --2- 172.21.15.70:0/1768933461 >> v2:172.21.15.44:3300/0 conn(0x7f7ae40c8430 0x7f7ae415ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:43.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.747+0000 7f7ae259c640 1 -- 172.21.15.70:0/1768933461 >> v2:172.21.15.70:3300/0 conn(0x7f7ae4144720 msgr2=0x7f7ae415f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:30:43.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.747+0000 7f7ae259c640 1 --2- 172.21.15.70:0/1768933461 >> v2:172.21.15.70:3300/0 conn(0x7f7ae4144720 0x7f7ae415f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:43.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.747+0000 7f7ae259c640 1 -- 172.21.15.70:0/1768933461 >> v2:172.21.15.44:3300/0 conn(0x7f7ae40c8430 msgr2=0x7f7ae415ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:30:43.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.747+0000 7f7ae259c640 1 --2- 172.21.15.70:0/1768933461 >> v2:172.21.15.44:3300/0 conn(0x7f7ae40c8430 0x7f7ae415ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:43.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.747+0000 7f7ae259c640 1 -- 172.21.15.70:0/1768933461 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ae4072c60 con 0x7f7ae4144340 2024-04-22T06:30:43.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.747+0000 7f7ae1d9b640 1 --2- 172.21.15.70:0/1768933461 >> v2:172.21.15.44:3300/0 conn(0x7f7ae40c8430 0x7f7ae415ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:30:43.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.748+0000 7f7ae259c640 1 --2- 172.21.15.70:0/1768933461 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ae4144340 0x7f7ae40756c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7ae406f050 tx=0x7f7ad402b9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a36860e5a5c77d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:30:43.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.748+0000 7f7ae159a640 1 -- 172.21.15.70:0/1768933461 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4034070 con 0x7f7ae4144340 2024-04-22T06:30:43.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.748+0000 7f7ae159a640 1 -- 172.21.15.70:0/1768933461 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ad4030860 con 0x7f7ae4144340 2024-04-22T06:30:43.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.748+0000 7f7ae159a640 1 -- 172.21.15.70:0/1768933461 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4030c30 con 0x7f7ae4144340 2024-04-22T06:30:43.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.749+0000 7f7ae8a43640 1 -- 172.21.15.70:0/1768933461 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ae4144340 msgr2=0x7f7ae40756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:43.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.749+0000 7f7ae8a43640 1 --2- 172.21.15.70:0/1768933461 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ae4144340 0x7f7ae40756c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7ae406f050 tx=0x7f7ad402b9a0 comp rx=0 tx=0).stop 2024-04-22T06:30:43.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.749+0000 7f7ae8a43640 1 -- 172.21.15.70:0/1768933461 shutdown_connections 2024-04-22T06:30:43.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.749+0000 7f7ae8a43640 1 --2- 172.21.15.70:0/1768933461 >> v2:172.21.15.70:3300/0 conn(0x7f7ae4144720 0x7f7ae415f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:43.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.749+0000 7f7ae8a43640 1 --2- 172.21.15.70:0/1768933461 >> v2:172.21.15.44:3300/0 conn(0x7f7ae40c8430 0x7f7ae415ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:43.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.749+0000 7f7ae8a43640 1 --2- 172.21.15.70:0/1768933461 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ae4144340 0x7f7ae40756c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:43.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.749+0000 7f7ae8a43640 1 -- 172.21.15.70:0/1768933461 >> 172.21.15.70:0/1768933461 conn(0x7f7ae40cd630 msgr2=0x7f7ae40cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:30:43.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.749+0000 7f7ae8a43640 1 -- 172.21.15.70:0/1768933461 shutdown_connections 2024-04-22T06:30:43.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.749+0000 7f7ae8a43640 1 -- 172.21.15.70:0/1768933461 wait complete. 2024-04-22T06:30:43.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.751+0000 7f7ae8a43640 1 Processor -- start 2024-04-22T06:30:43.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.751+0000 7f7ae8a43640 1 -- start start 2024-04-22T06:30:43.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.752+0000 7f7ae8a43640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ae40c8430 0x7f7ae40c7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:43.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.752+0000 7f7ae8a43640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ae4144340 0x7f7ae413da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:43.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.752+0000 7f7ae8a43640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ae4144720 0x7f7ae41f4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:43.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.752+0000 7f7ae8a43640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae406f9f0 con 0x7f7ae4144340 2024-04-22T06:30:43.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.752+0000 7f7ae8a43640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae406f870 con 0x7f7ae4144720 2024-04-22T06:30:43.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.752+0000 7f7ae8a43640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7ae406fb70 con 0x7f7ae40c8430 2024-04-22T06:30:43.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.752+0000 7f7ae259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ae40c8430 0x7f7ae40c7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:43.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.752+0000 7f7ae259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ae40c8430 0x7f7ae40c7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:48958/0 (socket says 172.21.15.70:48958) 2024-04-22T06:30:43.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.752+0000 7f7ae259c640 1 -- 172.21.15.70:0/444203365 learned_addr learned my addr 172.21.15.70:0/444203365 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-22T06:30:43.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.752+0000 7f7ae2d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ae4144720 0x7f7ae41f4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:43.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.753+0000 7f7ae1d9b640 1 --2- 172.21.15.70:0/444203365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ae4144340 0x7f7ae413da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:43.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.753+0000 7f7ae259c640 1 -- 172.21.15.70:0/444203365 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ae4144720 msgr2=0x7f7ae41f4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:43.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.753+0000 7f7ae259c640 1 --2- 172.21.15.70:0/444203365 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ae4144720 0x7f7ae41f4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:43.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.753+0000 7f7ae259c640 1 -- 172.21.15.70:0/444203365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ae4144340 msgr2=0x7f7ae413da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:43.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.753+0000 7f7ae259c640 1 --2- 172.21.15.70:0/444203365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ae4144340 0x7f7ae413da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:43.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.753+0000 7f7ae259c640 1 -- 172.21.15.70:0/444203365 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ae40c7830 con 0x7f7ae40c8430 2024-04-22T06:30:43.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.753+0000 7f7ae2d9d640 1 --2- 172.21.15.70:0/444203365 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ae4144720 0x7f7ae41f4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:30:43.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.753+0000 7f7ae259c640 1 --2- 172.21.15.70:0/444203365 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ae40c8430 0x7f7ae40c7240 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4000c00 tx=0x7f7ad4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:30:43.754 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T06:30:43.753+0000 7fc89d821040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-22T06:30:43.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.753+0000 7f7ad2ffd640 1 -- 172.21.15.70:0/444203365 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4048070 con 0x7f7ae40c8430 2024-04-22T06:30:43.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.753+0000 7f7ad2ffd640 1 -- 172.21.15.70:0/444203365 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ad4030660 con 0x7f7ae40c8430 2024-04-22T06:30:43.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.753+0000 7f7ae8a43640 1 -- 172.21.15.70:0/444203365 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ae40c5300 con 0x7f7ae40c8430 2024-04-22T06:30:43.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.753+0000 7f7ad2ffd640 1 -- 172.21.15.70:0/444203365 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4034040 con 0x7f7ae40c8430 2024-04-22T06:30:43.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.753+0000 7f7ae8a43640 1 -- 172.21.15.70:0/444203365 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ae40c58c0 con 0x7f7ae40c8430 2024-04-22T06:30:43.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.753+0000 7f7ad2ffd640 1 -- 172.21.15.70:0/444203365 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7ad404b070 con 0x7f7ae40c8430 2024-04-22T06:30:43.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.753+0000 7f7ad2ffd640 1 -- 172.21.15.70:0/444203365 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f7ad4043920 con 0x7f7ae40c8430 2024-04-22T06:30:43.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.754+0000 7f7ae8a43640 1 -- 172.21.15.70:0/444203365 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ae41552a0 con 0x7f7ae40c8430 2024-04-22T06:30:43.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.756+0000 7f7ad2ffd640 1 -- 172.21.15.70:0/444203365 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f7ae41552a0 con 0x7f7ae40c8430 2024-04-22T06:30:43.764 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T06:30:43.765+0000 7fba14012040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-22T06:30:43.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.829+0000 7f7ae8a43640 1 -- 172.21.15.70:0/444203365 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd new", "uuid": "357c439a-6e08-4c19-ac18-b68e6c890497", "id": 4} v 0) -- 0x7f7ae40c80f0 con 0x7f7ae40c8430 2024-04-22T06:30:43.831 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.832+0000 7f7ad2ffd640 1 -- 172.21.15.70:0/444203365 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "357c439a-6e08-4c19-ac18-b68e6c890497", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f7ad4052070 con 0x7f7ae40c8430 2024-04-22T06:30:43.831 INFO:teuthology.orchestra.run.smithi070.stdout:4 2024-04-22T06:30:43.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.833+0000 7f7ae8a43640 1 -- 172.21.15.70:0/444203365 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ae40c8430 msgr2=0x7f7ae40c7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:43.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.833+0000 7f7ae8a43640 1 --2- 172.21.15.70:0/444203365 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ae40c8430 0x7f7ae40c7240 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4000c00 tx=0x7f7ad4004730 comp rx=0 tx=0).stop 2024-04-22T06:30:43.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.833+0000 7f7ae8a43640 1 -- 172.21.15.70:0/444203365 shutdown_connections 2024-04-22T06:30:43.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.833+0000 7f7ae8a43640 1 --2- 172.21.15.70:0/444203365 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ae4144720 0x7f7ae41f4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:43.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.833+0000 7f7ae8a43640 1 --2- 172.21.15.70:0/444203365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ae4144340 0x7f7ae413da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:43.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.833+0000 7f7ae8a43640 1 --2- 172.21.15.70:0/444203365 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7ae40c8430 0x7f7ae40c7240 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:43.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.833+0000 7f7ae8a43640 1 -- 172.21.15.70:0/444203365 >> 172.21.15.70:0/444203365 conn(0x7f7ae40cd630 msgr2=0x7f7ae40cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:30:43.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.833+0000 7f7ae8a43640 1 -- 172.21.15.70:0/444203365 shutdown_connections 2024-04-22T06:30:43.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.833+0000 7f7ae8a43640 1 -- 172.21.15.70:0/444203365 wait complete. 2024-04-22T06:30:43.842 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd new 952fb88f-d56e-4305-bfa8-d32674332dfe 5 2024-04-22T06:30:43.845 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T06:30:43.845+0000 7fc89d821040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-22T06:30:43.856 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T06:30:43.858+0000 7fba14012040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-22T06:30:43.889 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T06:30:43.890+0000 7fc89d821040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-22T06:30:43.901 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T06:30:43.902+0000 7fba14012040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-22T06:30:43.938 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.939+0000 7f3ca06f2640 1 Processor -- start 2024-04-22T06:30:43.938 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.939+0000 7f3ca06f2640 1 -- start start 2024-04-22T06:30:43.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.940+0000 7f3ca06f2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3c981552b0 0x7f3c98067c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:43.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.940+0000 7f3ca06f2640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c9815bff0 con 0x7f3c98155690 2024-04-22T06:30:43.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.940+0000 7f3ca06f2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c9815c2d0 con 0x7f3c9806ed60 2024-04-22T06:30:43.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.940+0000 7f3ca06f2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f3c980c8910 con 0x7f3c981552b0 2024-04-22T06:30:43.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.940+0000 7f3c9e48e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3c981552b0 0x7f3c98067c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:43.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.940+0000 7f3c9e48e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3c981552b0 0x7f3c98067c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41456/0 (socket says 172.21.15.70:41456) 2024-04-22T06:30:43.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.940+0000 7f3c9dc8d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f3c9806ed60 0x7f3c980c8da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36924/0 (socket says 172.21.15.70:36924) 2024-04-22T06:30:43.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.940+0000 7f3c9e48e640 1 -- 172.21.15.70:0/3171716744 learned_addr learned my addr 172.21.15.70:0/3171716744 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-22T06:30:43.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.940+0000 7f3c9e48e640 1 -- 172.21.15.70:0/3171716744 >> v1:172.21.15.70:6789/0 conn(0x7f3c9806ed60 legacy=0x7f3c980c8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:43.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.940+0000 7f3c9e48e640 1 -- 172.21.15.70:0/3171716744 >> v1:172.21.15.44:6789/0 conn(0x7f3c98155690 legacy=0x7f3c98071aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:43.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.940+0000 7f3c9e48e640 1 -- 172.21.15.70:0/3171716744 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c98073f40 con 0x7f3c981552b0 2024-04-22T06:30:43.942 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.941+0000 7f3c9e48e640 1 --2- 172.21.15.70:0/3171716744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c981552b0 0x7f3c98067c40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3c88009870 tx=0x7f3c8802ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bb218bce425a4c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:30:43.942 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.941+0000 7f3c9d48c640 1 -- 172.21.15.70:0/3171716744 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8802f840 con 0x7f3c981552b0 2024-04-22T06:30:43.942 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.941+0000 7f3c9d48c640 1 -- 172.21.15.70:0/3171716744 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c8802f9e0 con 0x7f3c981552b0 2024-04-22T06:30:43.942 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.941+0000 7f3c9d48c640 1 -- 172.21.15.70:0/3171716744 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8802fdb0 con 0x7f3c981552b0 2024-04-22T06:30:43.942 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.942+0000 7f3ca06f2640 1 -- 172.21.15.70:0/3171716744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c981552b0 msgr2=0x7f3c98067c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:43.942 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.942+0000 7f3ca06f2640 1 --2- 172.21.15.70:0/3171716744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c981552b0 0x7f3c98067c40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3c88009870 tx=0x7f3c8802ba20 comp rx=0 tx=0).stop 2024-04-22T06:30:43.942 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.942+0000 7f3ca06f2640 1 -- 172.21.15.70:0/3171716744 shutdown_connections 2024-04-22T06:30:43.942 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.942+0000 7f3ca06f2640 1 --2- 172.21.15.70:0/3171716744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c981552b0 0x7f3c98067c40 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:43.942 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.942+0000 7f3ca06f2640 1 -- 172.21.15.70:0/3171716744 >> 172.21.15.70:0/3171716744 conn(0x7f3c980ccff0 msgr2=0x7f3c980cd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:30:43.942 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.942+0000 7f3ca06f2640 1 -- 172.21.15.70:0/3171716744 shutdown_connections 2024-04-22T06:30:43.942 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.942+0000 7f3ca06f2640 1 -- 172.21.15.70:0/3171716744 wait complete. 2024-04-22T06:30:43.947 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.943+0000 7f3ca06f2640 1 Processor -- start 2024-04-22T06:30:43.947 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.943+0000 7f3ca06f2640 1 -- start start 2024-04-22T06:30:43.947 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.944+0000 7f3ca06f2640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3c9806ed60 0x7f3c9815ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:43.947 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.944+0000 7f3ca06f2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c981552b0 0x7f3c98150d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:43.947 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.944+0000 7f3ca06f2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c98155690 0x7f3c981512c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:43.947 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.944+0000 7f3ca06f2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3c980c9630 con 0x7f3c98155690 2024-04-22T06:30:43.948 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.944+0000 7f3ca06f2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f3c980c94b0 con 0x7f3c981552b0 2024-04-22T06:30:43.948 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.944+0000 7f3ca06f2640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3c980b0520 con 0x7f3c9806ed60 2024-04-22T06:30:43.948 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.944+0000 7f3c9dc8d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c981552b0 0x7f3c98150d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:43.948 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.944+0000 7f3c9dc8d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c981552b0 0x7f3c98150d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41470/0 (socket says 172.21.15.70:41470) 2024-04-22T06:30:43.948 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.944+0000 7f3c9e48e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3c9806ed60 0x7f3c9815ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:43.948 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.944+0000 7f3c9dc8d640 1 -- 172.21.15.70:0/2039772699 learned_addr learned my addr 172.21.15.70:0/2039772699 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-22T06:30:43.948 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.944+0000 7f3c9ec8f640 1 --2- 172.21.15.70:0/2039772699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c98155690 0x7f3c981512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:43.948 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.944+0000 7f3c9dc8d640 1 -- 172.21.15.70:0/2039772699 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3c9806ed60 msgr2=0x7f3c9815ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:43.948 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.944+0000 7f3c9dc8d640 1 --2- 172.21.15.70:0/2039772699 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3c9806ed60 0x7f3c9815ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:43.948 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.944+0000 7f3c9dc8d640 1 -- 172.21.15.70:0/2039772699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c98155690 msgr2=0x7f3c981512c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:43.948 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.944+0000 7f3c9dc8d640 1 --2- 172.21.15.70:0/2039772699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c98155690 0x7f3c981512c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:43.948 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.944+0000 7f3c9dc8d640 1 -- 172.21.15.70:0/2039772699 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c980af980 con 0x7f3c981552b0 2024-04-22T06:30:43.948 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.944+0000 7f3c9e48e640 1 --2- 172.21.15.70:0/2039772699 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3c9806ed60 0x7f3c9815ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:30:43.948 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.944+0000 7f3c9dc8d640 1 --2- 172.21.15.70:0/2039772699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c981552b0 0x7f3c98150d80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3c9400c4d0 tx=0x7f3c9400c990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:30:43.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.944+0000 7f3c8effd640 1 -- 172.21.15.70:0/2039772699 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c94018070 con 0x7f3c981552b0 2024-04-22T06:30:43.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.944+0000 7f3c8effd640 1 -- 172.21.15.70:0/2039772699 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c94007480 con 0x7f3c981552b0 2024-04-22T06:30:43.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.944+0000 7f3c8effd640 1 -- 172.21.15.70:0/2039772699 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c940047e0 con 0x7f3c981552b0 2024-04-22T06:30:43.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.944+0000 7f3ca06f2640 1 -- 172.21.15.70:0/2039772699 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c9815f790 con 0x7f3c981552b0 2024-04-22T06:30:43.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.944+0000 7f3ca06f2640 1 -- 172.21.15.70:0/2039772699 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c980afed0 con 0x7f3c981552b0 2024-04-22T06:30:43.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.945+0000 7f3c8effd640 1 -- 172.21.15.70:0/2039772699 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3c9401b070 con 0x7f3c981552b0 2024-04-22T06:30:43.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.945+0000 7f3c8effd640 1 -- 172.21.15.70:0/2039772699 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f3c940129c0 con 0x7f3c981552b0 2024-04-22T06:30:43.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.946+0000 7f3ca06f2640 1 -- 172.21.15.70:0/2039772699 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c98072e00 con 0x7f3c981552b0 2024-04-22T06:30:43.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:43.948+0000 7f3c8effd640 1 -- 172.21.15.70:0/2039772699 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f3c98072e00 con 0x7f3c981552b0 2024-04-22T06:30:44.031 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.032+0000 7f3ca06f2640 1 -- 172.21.15.70:0/2039772699 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd new", "uuid": "952fb88f-d56e-4305-bfa8-d32674332dfe", "id": 5} v 0) -- 0x7f3c98069310 con 0x7f3c981552b0 2024-04-22T06:30:44.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.037+0000 7f3c8effd640 1 -- 172.21.15.70:0/2039772699 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "952fb88f-d56e-4305-bfa8-d32674332dfe", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f3c94022020 con 0x7f3c981552b0 2024-04-22T06:30:44.036 INFO:teuthology.orchestra.run.smithi070.stdout:5 2024-04-22T06:30:44.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.038+0000 7f3ca06f2640 1 -- 172.21.15.70:0/2039772699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c981552b0 msgr2=0x7f3c98150d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:44.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.038+0000 7f3ca06f2640 1 --2- 172.21.15.70:0/2039772699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c981552b0 0x7f3c98150d80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3c9400c4d0 tx=0x7f3c9400c990 comp rx=0 tx=0).stop 2024-04-22T06:30:44.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.038+0000 7f3ca06f2640 1 -- 172.21.15.70:0/2039772699 shutdown_connections 2024-04-22T06:30:44.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.038+0000 7f3ca06f2640 1 --2- 172.21.15.70:0/2039772699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c98155690 0x7f3c981512c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:44.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.038+0000 7f3ca06f2640 1 --2- 172.21.15.70:0/2039772699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c981552b0 0x7f3c98150d80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:44.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.038+0000 7f3ca06f2640 1 --2- 172.21.15.70:0/2039772699 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3c9806ed60 0x7f3c9815ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:44.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.038+0000 7f3ca06f2640 1 -- 172.21.15.70:0/2039772699 >> 172.21.15.70:0/2039772699 conn(0x7f3c980ccff0 msgr2=0x7f3c980cb670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:30:44.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.039+0000 7f3ca06f2640 1 -- 172.21.15.70:0/2039772699 shutdown_connections 2024-04-22T06:30:44.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.039+0000 7f3ca06f2640 1 -- 172.21.15.70:0/2039772699 wait complete. 2024-04-22T06:30:44.052 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd new 74399b3d-fac2-45ab-9172-2cd95b02450f 6 2024-04-22T06:30:44.076 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T06:30:44.077+0000 7fba14012040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-22T06:30:44.079 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T06:30:44.079+0000 7fc89d821040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-22T06:30:44.125 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T06:30:44.126+0000 7fba14012040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-22T06:30:44.138 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T06:30:44.139+0000 7fc89d821040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-22T06:30:44.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.149+0000 7f6841d04640 1 Processor -- start 2024-04-22T06:30:44.149 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.149+0000 7f6841d04640 1 -- start start 2024-04-22T06:30:44.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.150+0000 7f6841d04640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f683c0c9d20 0x7f683c15b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:44.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.150+0000 7f6841d04640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f683c072c20 0x7f683c15bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:44.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.150+0000 7f6841d04640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f683c157d70 con 0x7f683c151fa0 2024-04-22T06:30:44.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.150+0000 7f6841d04640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f683c15b360 con 0x7f683c072c20 2024-04-22T06:30:44.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.150+0000 7f6841d04640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f683c158030 con 0x7f683c0c9d20 2024-04-22T06:30:44.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.150+0000 7f683bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f683c072c20 0x7f683c15bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:44.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.150+0000 7f6833fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f683c0c9d20 0x7f683c15b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:44.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.150+0000 7f683b7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f683c151fa0 0x7f683c0cd5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36926/0 (socket says 172.21.15.70:36926) 2024-04-22T06:30:44.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.150+0000 7f683b7fe640 1 -- 172.21.15.70:0/541433641 learned_addr learned my addr 172.21.15.70:0/541433641 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-04-22T06:30:44.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.150+0000 7f6833fff640 1 -- 172.21.15.70:0/541433641 >> v2:172.21.15.70:3300/0 conn(0x7f683c072c20 msgr2=0x7f683c15bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:30:44.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.150+0000 7f6833fff640 1 --2- 172.21.15.70:0/541433641 >> v2:172.21.15.70:3300/0 conn(0x7f683c072c20 0x7f683c15bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:44.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.150+0000 7f6833fff640 1 -- 172.21.15.70:0/541433641 >> v1:172.21.15.70:6789/0 conn(0x7f683c151fa0 legacy=0x7f683c0cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:44.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.150+0000 7f6833fff640 1 -- 172.21.15.70:0/541433641 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f683c158310 con 0x7f683c0c9d20 2024-04-22T06:30:44.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.150+0000 7f683affd640 1 -- 172.21.15.70:0/541433641 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2960559969 0 0) 0x7f683c157d70 con 0x7f683c151fa0 2024-04-22T06:30:44.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.151+0000 7f683bfff640 1 --2- 172.21.15.70:0/541433641 >> v2:172.21.15.70:3300/0 conn(0x7f683c072c20 0x7f683c15bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:30:44.152 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.151+0000 7f6833fff640 1 --2- 172.21.15.70:0/541433641 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f683c0c9d20 0x7f683c15b7f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f68240097a0 tx=0x7f682402b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e1f33066c21d666 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:30:44.152 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.151+0000 7f683affd640 1 -- 172.21.15.70:0/541433641 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6824034070 con 0x7f683c0c9d20 2024-04-22T06:30:44.152 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.151+0000 7f683affd640 1 -- 172.21.15.70:0/541433641 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6824030610 con 0x7f683c0c9d20 2024-04-22T06:30:44.152 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.151+0000 7f683affd640 1 -- 172.21.15.70:0/541433641 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68240309e0 con 0x7f683c0c9d20 2024-04-22T06:30:44.152 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.152+0000 7f6841d04640 1 -- 172.21.15.70:0/541433641 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f683c0c9d20 msgr2=0x7f683c15b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:44.152 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.152+0000 7f6841d04640 1 --2- 172.21.15.70:0/541433641 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f683c0c9d20 0x7f683c15b7f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f68240097a0 tx=0x7f682402b960 comp rx=0 tx=0).stop 2024-04-22T06:30:44.152 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.152+0000 7f6841d04640 1 -- 172.21.15.70:0/541433641 shutdown_connections 2024-04-22T06:30:44.153 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.152+0000 7f6841d04640 1 --2- 172.21.15.70:0/541433641 >> v2:172.21.15.70:3300/0 conn(0x7f683c072c20 0x7f683c15bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:44.153 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.152+0000 7f6841d04640 1 --2- 172.21.15.70:0/541433641 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f683c0c9d20 0x7f683c15b7f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:44.153 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.152+0000 7f6841d04640 1 -- 172.21.15.70:0/541433641 >> 172.21.15.70:0/541433641 conn(0x7f683c06fe00 msgr2=0x7f683c0701e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:30:44.153 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.152+0000 7f6841d04640 1 -- 172.21.15.70:0/541433641 shutdown_connections 2024-04-22T06:30:44.153 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.152+0000 7f6841d04640 1 -- 172.21.15.70:0/541433641 wait complete. 2024-04-22T06:30:44.153 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.154+0000 7f6841d04640 1 Processor -- start 2024-04-22T06:30:44.153 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.154+0000 7f6841d04640 1 -- start start 2024-04-22T06:30:44.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.155+0000 7f6841d04640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f683c072c20 0x7f683c1568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:44.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.155+0000 7f6841d04640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f683c0c9d20 0x7f683c156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:44.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.155+0000 7f6841d04640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f683c151fa0 0x7f683c0c6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:44.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.155+0000 7f6841d04640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f683c158bb0 con 0x7f683c0c9d20 2024-04-22T06:30:44.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.155+0000 7f6841d04640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f683c1588d0 con 0x7f683c072c20 2024-04-22T06:30:44.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.155+0000 7f6841d04640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f683c152380 con 0x7f683c151fa0 2024-04-22T06:30:44.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.155+0000 7f683bfff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f683c151fa0 0x7f683c0c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:44.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.155+0000 7f683b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f683c072c20 0x7f683c1568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:44.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.155+0000 7f683bfff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f683c151fa0 0x7f683c0c6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:48988/0 (socket says 172.21.15.70:48988) 2024-04-22T06:30:44.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.155+0000 7f683bfff640 1 -- 172.21.15.70:0/3929701432 learned_addr learned my addr 172.21.15.70:0/3929701432 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-22T06:30:44.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.155+0000 7f6833fff640 1 --2- 172.21.15.70:0/3929701432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f683c0c9d20 0x7f683c156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:44.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.155+0000 7f683b7fe640 1 -- 172.21.15.70:0/3929701432 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f683c151fa0 msgr2=0x7f683c0c6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:44.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.155+0000 7f683b7fe640 1 --2- 172.21.15.70:0/3929701432 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f683c151fa0 0x7f683c0c6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:44.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.155+0000 7f683b7fe640 1 -- 172.21.15.70:0/3929701432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f683c0c9d20 msgr2=0x7f683c156e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:44.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.155+0000 7f683b7fe640 1 --2- 172.21.15.70:0/3929701432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f683c0c9d20 0x7f683c156e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:44.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.155+0000 7f683b7fe640 1 -- 172.21.15.70:0/3929701432 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f683c0c7750 con 0x7f683c072c20 2024-04-22T06:30:44.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.155+0000 7f683bfff640 1 --2- 172.21.15.70:0/3929701432 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f683c151fa0 0x7f683c0c6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:30:44.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.155+0000 7f6833fff640 1 --2- 172.21.15.70:0/3929701432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f683c0c9d20 0x7f683c156e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:30:44.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.156+0000 7f683b7fe640 1 --2- 172.21.15.70:0/3929701432 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f683c072c20 0x7f683c1568d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f683c070cc0 tx=0x7f682c00c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:30:44.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.156+0000 7f6838ff9640 1 -- 172.21.15.70:0/3929701432 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c017070 con 0x7f683c072c20 2024-04-22T06:30:44.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.156+0000 7f6838ff9640 1 -- 172.21.15.70:0/3929701432 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f682c005ce0 con 0x7f683c072c20 2024-04-22T06:30:44.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.156+0000 7f6841d04640 1 -- 172.21.15.70:0/3929701432 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f683c1ed6f0 con 0x7f683c072c20 2024-04-22T06:30:44.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.156+0000 7f6838ff9640 1 -- 172.21.15.70:0/3929701432 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c007310 con 0x7f683c072c20 2024-04-22T06:30:44.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.156+0000 7f6841d04640 1 -- 172.21.15.70:0/3929701432 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f683c156110 con 0x7f683c072c20 2024-04-22T06:30:44.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.156+0000 7f6838ff9640 1 -- 172.21.15.70:0/3929701432 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f682c01a070 con 0x7f683c072c20 2024-04-22T06:30:44.158 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.156+0000 7f6838ff9640 1 -- 172.21.15.70:0/3929701432 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f682c0129f0 con 0x7f683c072c20 2024-04-22T06:30:44.158 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.157+0000 7f6841d04640 1 -- 172.21.15.70:0/3929701432 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f683c069310 con 0x7f683c072c20 2024-04-22T06:30:44.158 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.159+0000 7f6838ff9640 1 -- 172.21.15.70:0/3929701432 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f682c021020 con 0x7f683c072c20 2024-04-22T06:30:44.180 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T06:30:44.182+0000 7fba14012040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-22T06:30:44.194 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T06:30:44.195+0000 7fc89d821040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-22T06:30:44.229 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.230+0000 7f6841d04640 1 -- 172.21.15.70:0/3929701432 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd new", "uuid": "74399b3d-fac2-45ab-9172-2cd95b02450f", "id": 6} v 0) -- 0x7f683c0c9330 con 0x7f683c072c20 2024-04-22T06:30:44.234 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.235+0000 7f6838ff9640 1 -- 172.21.15.70:0/3929701432 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "74399b3d-fac2-45ab-9172-2cd95b02450f", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f682c007580 con 0x7f683c072c20 2024-04-22T06:30:44.235 INFO:teuthology.orchestra.run.smithi070.stdout:6 2024-04-22T06:30:44.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.236+0000 7f6841d04640 1 -- 172.21.15.70:0/3929701432 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f683c072c20 msgr2=0x7f683c1568d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:44.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.236+0000 7f6841d04640 1 --2- 172.21.15.70:0/3929701432 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f683c072c20 0x7f683c1568d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f683c070cc0 tx=0x7f682c00c870 comp rx=0 tx=0).stop 2024-04-22T06:30:44.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.236+0000 7f6841d04640 1 -- 172.21.15.70:0/3929701432 shutdown_connections 2024-04-22T06:30:44.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.236+0000 7f6841d04640 1 --2- 172.21.15.70:0/3929701432 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f683c151fa0 0x7f683c0c6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:44.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.236+0000 7f6841d04640 1 --2- 172.21.15.70:0/3929701432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f683c0c9d20 0x7f683c156e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:44.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.236+0000 7f6841d04640 1 --2- 172.21.15.70:0/3929701432 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f683c072c20 0x7f683c1568d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:44.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.236+0000 7f6841d04640 1 -- 172.21.15.70:0/3929701432 >> 172.21.15.70:0/3929701432 conn(0x7f683c06fe00 msgr2=0x7f683c06c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:30:44.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.236+0000 7f6841d04640 1 -- 172.21.15.70:0/3929701432 shutdown_connections 2024-04-22T06:30:44.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.236+0000 7f6841d04640 1 -- 172.21.15.70:0/3929701432 wait complete. 2024-04-22T06:30:44.246 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd new 8f35508d-6fcc-43ff-8ee7-0498a7ab3b8d 7 2024-04-22T06:30:44.299 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T06:30:44.300+0000 7fba14012040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-22T06:30:44.311 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T06:30:44.312+0000 7fc89d821040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-22T06:30:44.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.385+0000 7f387a17e640 1 Processor -- start 2024-04-22T06:30:44.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.385+0000 7f387a17e640 1 -- start start 2024-04-22T06:30:44.386 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.385+0000 7f387a17e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f38740c83f0 0x7f387415baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:44.386 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.385+0000 7f387a17e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3874074cb0 0x7f3874075160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:44.386 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.385+0000 7f387a17e640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3874143f30 con 0x7f38741437f0 2024-04-22T06:30:44.386 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.385+0000 7f387a17e640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f3874144760 con 0x7f38740c83f0 2024-04-22T06:30:44.386 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.385+0000 7f387a17e640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f38740757b0 con 0x7f3874074cb0 2024-04-22T06:30:44.386 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.386+0000 7f3873fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3874074cb0 0x7f3874075160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:44.386 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.386+0000 7f3873fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3874074cb0 0x7f3874075160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:48996/0 (socket says 172.21.15.70:48996) 2024-04-22T06:30:44.386 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.386+0000 7f3873fff640 1 -- 172.21.15.70:0/653050229 learned_addr learned my addr 172.21.15.70:0/653050229 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-22T06:30:44.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.386+0000 7f3872ffd640 1 --2- 172.21.15.70:0/653050229 >> v2:172.21.15.44:3300/0 conn(0x7f38740c83f0 0x7f387415baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:44.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.386+0000 7f3873fff640 1 -- 172.21.15.70:0/653050229 >> v2:172.21.15.44:3300/0 conn(0x7f38740c83f0 msgr2=0x7f387415baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:30:44.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.386+0000 7f3873fff640 1 --2- 172.21.15.70:0/653050229 >> v2:172.21.15.44:3300/0 conn(0x7f38740c83f0 0x7f387415baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:44.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.386+0000 7f3873fff640 1 -- 172.21.15.70:0/653050229 >> v1:172.21.15.44:6789/0 conn(0x7f38741437f0 legacy=0x7f387415f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:44.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.386+0000 7f3873fff640 1 -- 172.21.15.70:0/653050229 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38741441a0 con 0x7f3874074cb0 2024-04-22T06:30:44.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.387+0000 7f3873fff640 1 --2- 172.21.15.70:0/653050229 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3874074cb0 0x7f3874075160 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f386c009e50 tx=0x7f386c02bff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa5add245d716cfd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:30:44.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.387+0000 7f38727fc640 1 -- 172.21.15.70:0/653050229 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386c02cb90 con 0x7f3874074cb0 2024-04-22T06:30:44.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.387+0000 7f38727fc640 1 -- 172.21.15.70:0/653050229 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f386c02cd30 con 0x7f3874074cb0 2024-04-22T06:30:44.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.387+0000 7f38727fc640 1 -- 172.21.15.70:0/653050229 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386c034610 con 0x7f3874074cb0 2024-04-22T06:30:44.390 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.387+0000 7f387a17e640 1 -- 172.21.15.70:0/653050229 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3874074cb0 msgr2=0x7f3874075160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:44.390 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.387+0000 7f387a17e640 1 --2- 172.21.15.70:0/653050229 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3874074cb0 0x7f3874075160 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f386c009e50 tx=0x7f386c02bff0 comp rx=0 tx=0).stop 2024-04-22T06:30:44.390 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.387+0000 7f387a17e640 1 -- 172.21.15.70:0/653050229 shutdown_connections 2024-04-22T06:30:44.390 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.387+0000 7f387a17e640 1 --2- 172.21.15.70:0/653050229 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3874074cb0 0x7f3874075160 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:44.390 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.387+0000 7f387a17e640 1 --2- 172.21.15.70:0/653050229 >> v2:172.21.15.44:3300/0 conn(0x7f38740c83f0 0x7f387415baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:44.390 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.387+0000 7f387a17e640 1 -- 172.21.15.70:0/653050229 >> 172.21.15.70:0/653050229 conn(0x7f38740cd630 msgr2=0x7f38740cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:30:44.390 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.387+0000 7f387a17e640 1 -- 172.21.15.70:0/653050229 shutdown_connections 2024-04-22T06:30:44.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.387+0000 7f387a17e640 1 -- 172.21.15.70:0/653050229 wait complete. 2024-04-22T06:30:44.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.388+0000 7f387a17e640 1 Processor -- start 2024-04-22T06:30:44.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.389+0000 7f387a17e640 1 -- start start 2024-04-22T06:30:44.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.389+0000 7f387a17e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3874074cb0 0x7f38741393b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:44.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.389+0000 7f387a17e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38740c83f0 0x7f3874141910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:44.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.389+0000 7f387a17e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38741437f0 0x7f3874141e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:44.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.389+0000 7f387a17e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f387415b5d0 con 0x7f38740c83f0 2024-04-22T06:30:44.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.389+0000 7f387a17e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f387415b2f0 con 0x7f3874074cb0 2024-04-22T06:30:44.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.389+0000 7f387a17e640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f387415c030 con 0x7f38741437f0 2024-04-22T06:30:44.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.389+0000 7f38737fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3874074cb0 0x7f38741393b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:44.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.389+0000 7f3873fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38741437f0 0x7f3874141e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:44.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.389+0000 7f38737fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3874074cb0 0x7f38741393b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51706/0 (socket says 172.21.15.70:51706) 2024-04-22T06:30:44.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.389+0000 7f38737fe640 1 -- 172.21.15.70:0/2489907012 learned_addr learned my addr 172.21.15.70:0/2489907012 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-22T06:30:44.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.389+0000 7f3872ffd640 1 --2- 172.21.15.70:0/2489907012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38740c83f0 0x7f3874141910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:44.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.390+0000 7f38737fe640 1 -- 172.21.15.70:0/2489907012 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38741437f0 msgr2=0x7f3874141e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:44.392 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.390+0000 7f38737fe640 1 --2- 172.21.15.70:0/2489907012 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38741437f0 0x7f3874141e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:44.392 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.390+0000 7f38737fe640 1 -- 172.21.15.70:0/2489907012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38740c83f0 msgr2=0x7f3874141910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:44.392 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.390+0000 7f38737fe640 1 --2- 172.21.15.70:0/2489907012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38740c83f0 0x7f3874141910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:44.392 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.390+0000 7f38737fe640 1 -- 172.21.15.70:0/2489907012 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f387415ce40 con 0x7f3874074cb0 2024-04-22T06:30:44.392 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.390+0000 7f3873fff640 1 --2- 172.21.15.70:0/2489907012 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38741437f0 0x7f3874141e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:30:44.392 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.390+0000 7f38737fe640 1 --2- 172.21.15.70:0/2489907012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3874074cb0 0x7f38741393b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f386400c3f0 tx=0x7f386400c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:30:44.392 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.390+0000 7f385bfff640 1 -- 172.21.15.70:0/2489907012 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38640052d0 con 0x7f3874074cb0 2024-04-22T06:30:44.392 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.390+0000 7f385bfff640 1 -- 172.21.15.70:0/2489907012 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3864005490 con 0x7f3874074cb0 2024-04-22T06:30:44.392 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.390+0000 7f387a17e640 1 -- 172.21.15.70:0/2489907012 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38741398f0 con 0x7f3874074cb0 2024-04-22T06:30:44.392 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.390+0000 7f385bfff640 1 -- 172.21.15.70:0/2489907012 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3864007260 con 0x7f3874074cb0 2024-04-22T06:30:44.392 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.390+0000 7f387a17e640 1 -- 172.21.15.70:0/2489907012 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f387415d3a0 con 0x7f3874074cb0 2024-04-22T06:30:44.392 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.390+0000 7f385bfff640 1 -- 172.21.15.70:0/2489907012 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3864007400 con 0x7f3874074cb0 2024-04-22T06:30:44.392 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.390+0000 7f385bfff640 1 -- 172.21.15.70:0/2489907012 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f3864011bf0 con 0x7f3874074cb0 2024-04-22T06:30:44.393 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.391+0000 7f387a17e640 1 -- 172.21.15.70:0/2489907012 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3874069310 con 0x7f3874074cb0 2024-04-22T06:30:44.393 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.393+0000 7f385bfff640 1 -- 172.21.15.70:0/2489907012 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f38640035f0 con 0x7f3874074cb0 2024-04-22T06:30:44.464 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.465+0000 7f387a17e640 1 -- 172.21.15.70:0/2489907012 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8f35508d-6fcc-43ff-8ee7-0498a7ab3b8d", "id": 7} v 0) -- 0x7f38740c7a30 con 0x7f3874074cb0 2024-04-22T06:30:44.469 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.470+0000 7f385bfff640 1 -- 172.21.15.70:0/2489907012 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8f35508d-6fcc-43ff-8ee7-0498a7ab3b8d", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f3864003a00 con 0x7f3874074cb0 2024-04-22T06:30:44.469 INFO:teuthology.orchestra.run.smithi070.stdout:7 2024-04-22T06:30:44.470 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.471+0000 7f387a17e640 1 -- 172.21.15.70:0/2489907012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3874074cb0 msgr2=0x7f38741393b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:44.471 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.471+0000 7f387a17e640 1 --2- 172.21.15.70:0/2489907012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3874074cb0 0x7f38741393b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f386400c3f0 tx=0x7f386400c8b0 comp rx=0 tx=0).stop 2024-04-22T06:30:44.471 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.471+0000 7f387a17e640 1 -- 172.21.15.70:0/2489907012 shutdown_connections 2024-04-22T06:30:44.471 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.471+0000 7f387a17e640 1 --2- 172.21.15.70:0/2489907012 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38741437f0 0x7f3874141e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:44.471 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.471+0000 7f387a17e640 1 --2- 172.21.15.70:0/2489907012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38740c83f0 0x7f3874141910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:44.471 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.471+0000 7f387a17e640 1 --2- 172.21.15.70:0/2489907012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3874074cb0 0x7f38741393b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:44.471 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.471+0000 7f387a17e640 1 -- 172.21.15.70:0/2489907012 >> 172.21.15.70:0/2489907012 conn(0x7f38740cd630 msgr2=0x7f3874073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:30:44.471 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.471+0000 7f387a17e640 1 -- 172.21.15.70:0/2489907012 shutdown_connections 2024-04-22T06:30:44.471 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T06:30:44.471+0000 7f387a17e640 1 -- 172.21.15.70:0/2489907012 wait complete. 2024-04-22T06:30:44.481 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-22T06:30:44.481 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-22T06:30:44.483 INFO:tasks.ceph.osd.0:Started 2024-04-22T06:30:44.483 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-22T06:30:44.483 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-22T06:30:44.487 INFO:tasks.ceph.osd.1:Started 2024-04-22T06:30:44.487 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-22T06:30:44.487 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-22T06:30:44.489 INFO:tasks.ceph.osd.2:Started 2024-04-22T06:30:44.489 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-22T06:30:44.489 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-22T06:30:44.491 INFO:tasks.ceph.osd.3:Started 2024-04-22T06:30:44.492 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-22T06:30:44.492 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-22T06:30:44.523 INFO:tasks.ceph.osd.4:Started 2024-04-22T06:30:44.523 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-22T06:30:44.524 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-22T06:30:44.526 INFO:tasks.ceph.osd.5:Started 2024-04-22T06:30:44.526 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-22T06:30:44.526 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-22T06:30:44.528 INFO:tasks.ceph.osd.6:Started 2024-04-22T06:30:44.528 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-22T06:30:44.528 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-22T06:30:44.530 INFO:tasks.ceph.osd.7:Started 2024-04-22T06:30:44.531 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T06:30:44.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.648+0000 7fc096ca5640 1 Processor -- start 2024-04-22T06:30:44.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.648+0000 7fc096ca5640 1 -- start start 2024-04-22T06:30:44.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.649+0000 7fc096ca5640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc090154e70 0x7fc0900cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:44.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.649+0000 7fc096ca5640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc0900ce070 0x7fc0900ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:44.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.649+0000 7fc096ca5640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0900ce520 con 0x7fc090072b10 2024-04-22T06:30:44.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.649+0000 7fc096ca5640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fc09015b320 con 0x7fc0900ce070 2024-04-22T06:30:44.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.649+0000 7fc096ca5640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc09015b600 con 0x7fc090154e70 2024-04-22T06:30:44.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.649+0000 7fc08ffff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc0900ce070 0x7fc0900ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:44.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.649+0000 7fc08ffff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc0900ce070 0x7fc0900ca0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57488/0 (socket says 172.21.15.44:57488) 2024-04-22T06:30:44.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.649+0000 7fc08ffff640 1 -- 172.21.15.44:0/2559609078 learned_addr learned my addr 172.21.15.44:0/2559609078 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:30:44.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.649+0000 7fc094a41640 1 --2- 172.21.15.44:0/2559609078 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc090154e70 0x7fc0900cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:44.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.649+0000 7fc08ffff640 1 -- 172.21.15.44:0/2559609078 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc090154e70 msgr2=0x7fc0900cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:30:44.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.649+0000 7fc08ffff640 1 --2- 172.21.15.44:0/2559609078 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc090154e70 0x7fc0900cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:44.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.649+0000 7fc08ffff640 1 -- 172.21.15.44:0/2559609078 >> v1:172.21.15.70:6789/0 conn(0x7fc090072b10 legacy=0x7fc09015e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:44.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.649+0000 7fc08ffff640 1 -- 172.21.15.44:0/2559609078 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc09015b8e0 con 0x7fc0900ce070 2024-04-22T06:30:44.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.650+0000 7fc08ffff640 1 --2- 172.21.15.44:0/2559609078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0900ce070 0x7fc0900ca0e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc07c009870 tx=0x7fc07c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13847de070f29bd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:30:44.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.650+0000 7fc08f7fe640 1 -- 172.21.15.44:0/2559609078 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07c035020 con 0x7fc0900ce070 2024-04-22T06:30:44.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.650+0000 7fc08f7fe640 1 -- 172.21.15.44:0/2559609078 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc07c031640 con 0x7fc0900ce070 2024-04-22T06:30:44.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.650+0000 7fc08f7fe640 1 -- 172.21.15.44:0/2559609078 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07c02d040 con 0x7fc0900ce070 2024-04-22T06:30:44.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.651+0000 7fc096ca5640 1 -- 172.21.15.44:0/2559609078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0900ce070 msgr2=0x7fc0900ca0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:44.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.651+0000 7fc096ca5640 1 --2- 172.21.15.44:0/2559609078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0900ce070 0x7fc0900ca0e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc07c009870 tx=0x7fc07c02b8f0 comp rx=0 tx=0).stop 2024-04-22T06:30:44.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.651+0000 7fc096ca5640 1 -- 172.21.15.44:0/2559609078 shutdown_connections 2024-04-22T06:30:44.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.651+0000 7fc096ca5640 1 --2- 172.21.15.44:0/2559609078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0900ce070 0x7fc0900ca0e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:44.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.651+0000 7fc096ca5640 1 --2- 172.21.15.44:0/2559609078 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc090154e70 0x7fc0900cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:44.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.651+0000 7fc096ca5640 1 -- 172.21.15.44:0/2559609078 >> 172.21.15.44:0/2559609078 conn(0x7fc090067f80 msgr2=0x7fc09006fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:30:44.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.651+0000 7fc096ca5640 1 -- 172.21.15.44:0/2559609078 shutdown_connections 2024-04-22T06:30:44.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.651+0000 7fc096ca5640 1 -- 172.21.15.44:0/2559609078 wait complete. 2024-04-22T06:30:44.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.652+0000 7fc096ca5640 1 Processor -- start 2024-04-22T06:30:44.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.652+0000 7fc096ca5640 1 -- start start 2024-04-22T06:30:44.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.653+0000 7fc096ca5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc090072b10 0x7fc09014daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:44.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.653+0000 7fc096ca5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0900ce070 0x7fc0901f2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:44.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.653+0000 7fc096ca5640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc090154e70 0x7fc0901f4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:44.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.653+0000 7fc096ca5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc09015afd0 con 0x7fc090072b10 2024-04-22T06:30:44.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.653+0000 7fc096ca5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc09015bea0 con 0x7fc0900ce070 2024-04-22T06:30:44.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.653+0000 7fc096ca5640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc090155250 con 0x7fc090154e70 2024-04-22T06:30:44.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.653+0000 7fc094a41640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc090072b10 0x7fc09014daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:44.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.653+0000 7fc094a41640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc090072b10 0x7fc09014daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57496/0 (socket says 172.21.15.44:57496) 2024-04-22T06:30:44.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.653+0000 7fc094a41640 1 -- 172.21.15.44:0/3787867795 learned_addr learned my addr 172.21.15.44:0/3787867795 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:30:44.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.653+0000 7fc08ffff640 1 --2- 172.21.15.44:0/3787867795 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0900ce070 0x7fc0901f2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:44.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.653+0000 7fc095242640 1 --2- 172.21.15.44:0/3787867795 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc090154e70 0x7fc0901f4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:44.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.653+0000 7fc094a41640 1 -- 172.21.15.44:0/3787867795 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc090154e70 msgr2=0x7fc0901f4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:44.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.653+0000 7fc094a41640 1 --2- 172.21.15.44:0/3787867795 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc090154e70 0x7fc0901f4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:44.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.653+0000 7fc094a41640 1 -- 172.21.15.44:0/3787867795 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0900ce070 msgr2=0x7fc0901f2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:44.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.653+0000 7fc094a41640 1 --2- 172.21.15.44:0/3787867795 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0900ce070 0x7fc0901f2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:44.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.653+0000 7fc094a41640 1 -- 172.21.15.44:0/3787867795 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc09014e5c0 con 0x7fc090072b10 2024-04-22T06:30:44.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.653+0000 7fc094a41640 1 --2- 172.21.15.44:0/3787867795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc090072b10 0x7fc09014daf0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc08400e9b0 tx=0x7fc08400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:30:44.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.654+0000 7fc08d7fa640 1 -- 172.21.15.44:0/3787867795 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc084017070 con 0x7fc090072b10 2024-04-22T06:30:44.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.654+0000 7fc08d7fa640 1 -- 172.21.15.44:0/3787867795 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc08400c9a0 con 0x7fc090072b10 2024-04-22T06:30:44.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.654+0000 7fc096ca5640 1 -- 172.21.15.44:0/3787867795 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc090157270 con 0x7fc090072b10 2024-04-22T06:30:44.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.654+0000 7fc08d7fa640 1 -- 172.21.15.44:0/3787867795 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08400cd90 con 0x7fc090072b10 2024-04-22T06:30:44.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.654+0000 7fc096ca5640 1 -- 172.21.15.44:0/3787867795 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc090157830 con 0x7fc090072b10 2024-04-22T06:30:44.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.654+0000 7fc08d7fa640 1 -- 172.21.15.44:0/3787867795 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc08401a070 con 0x7fc090072b10 2024-04-22T06:30:44.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.654+0000 7fc08d7fa640 1 -- 172.21.15.44:0/3787867795 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fc084006660 con 0x7fc090072b10 2024-04-22T06:30:44.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.655+0000 7fc096ca5640 1 -- 172.21.15.44:0/3787867795 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc090073de0 con 0x7fc090072b10 2024-04-22T06:30:44.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.657+0000 7fc08d7fa640 1 -- 172.21.15.44:0/3787867795 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fc090073de0 con 0x7fc090072b10 2024-04-22T06:30:44.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.719+0000 7fc096ca5640 1 -- 172.21.15.44:0/3787867795 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc090069310 con 0x7fc090072b10 2024-04-22T06:30:44.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.720+0000 7fc08d7fa640 1 -- 172.21.15.44:0/3787867795 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fc084005990 con 0x7fc090072b10 2024-04-22T06:30:44.719 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:30:44.719 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":10,"fsid":"9b0a34ca-403d-4513-8a66-d3749ba559aa","created":"2024-04-22T06:30:42.334581+0000","modified":"2024-04-22T06:30:44.467727+0000","last_up_change":"0.000000","last_in_change":"2024-04-22T06:30:44.467727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2b4a6692-53c9-472d-b2f3-1dd29a6679ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"70a6370a-7c0a-4d64-a0e4-b2e726b4f291","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f2e51a62-ca91-470d-bd5a-5710c61deee4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"bc514203-9af9-4f1f-b18e-765bdd5de6e7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"357c439a-6e08-4c19-ac18-b68e6c890497","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"952fb88f-d56e-4305-bfa8-d32674332dfe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"74399b3d-fac2-45ab-9172-2cd95b02450f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"8f35508d-6fcc-43ff-8ee7-0498a7ab3b8d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T06:30:44.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.721+0000 7fc096ca5640 1 -- 172.21.15.44:0/3787867795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc090072b10 msgr2=0x7fc09014daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:44.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.721+0000 7fc096ca5640 1 --2- 172.21.15.44:0/3787867795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc090072b10 0x7fc09014daf0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc08400e9b0 tx=0x7fc08400ee70 comp rx=0 tx=0).stop 2024-04-22T06:30:44.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.721+0000 7fc096ca5640 1 -- 172.21.15.44:0/3787867795 shutdown_connections 2024-04-22T06:30:44.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.721+0000 7fc096ca5640 1 --2- 172.21.15.44:0/3787867795 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc090154e70 0x7fc0901f4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:44.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.721+0000 7fc096ca5640 1 --2- 172.21.15.44:0/3787867795 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0900ce070 0x7fc0901f2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:44.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.721+0000 7fc096ca5640 1 --2- 172.21.15.44:0/3787867795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc090072b10 0x7fc09014daf0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:44.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.721+0000 7fc096ca5640 1 -- 172.21.15.44:0/3787867795 >> 172.21.15.44:0/3787867795 conn(0x7fc090067f80 msgr2=0x7fc09015e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:30:44.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.721+0000 7fc096ca5640 1 -- 172.21.15.44:0/3787867795 shutdown_connections 2024-04-22T06:30:44.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:30:44.721+0000 7fc096ca5640 1 -- 172.21.15.44:0/3787867795 wait complete. 2024-04-22T06:30:44.727 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T06:30:44.727+0000 7fc89d821040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-22T06:30:44.730 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-22T06:30:44.730 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-22T06:30:44.735 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T06:30:44.737+0000 7fba14012040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-22T06:30:44.985 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T06:30:44.987+0000 7fc89d821040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-22T06:30:45.013 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T06:30:45.014+0000 7fba14012040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-22T06:30:45.036 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T06:30:45.038+0000 7fc89d821040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-22T06:30:45.069 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T06:30:45.070+0000 7fba14012040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-22T06:30:45.090 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-04-22T06:30:45.092+0000 7f4c4d814640 -1 Falling back to public interface 2024-04-22T06:30:45.104 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-04-22T06:30:45.106+0000 7fa72b413640 -1 Falling back to public interface 2024-04-22T06:30:45.118 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-22T06:30:45.119+0000 7f0514323640 -1 Falling back to public interface 2024-04-22T06:30:45.131 INFO:tasks.ceph.osd.3.smithi044.stderr:2024-04-22T06:30:45.132+0000 7f255f015640 -1 Falling back to public interface 2024-04-22T06:30:45.131 INFO:tasks.ceph.osd.5.smithi070.stderr:2024-04-22T06:30:45.133+0000 7fb039213640 -1 Falling back to public interface 2024-04-22T06:30:45.135 INFO:tasks.ceph.osd.7.smithi070.stderr:2024-04-22T06:30:45.137+0000 7ff5f601f640 -1 Falling back to public interface 2024-04-22T06:30:45.140 INFO:tasks.ceph.osd.6.smithi070.stderr:2024-04-22T06:30:45.142+0000 7f1fe47dc640 -1 Falling back to public interface 2024-04-22T06:30:45.141 INFO:tasks.ceph.osd.4.smithi070.stderr:2024-04-22T06:30:45.142+0000 7f0596625640 -1 Falling back to public interface 2024-04-22T06:30:45.141 INFO:tasks.ceph.osd.4.smithi070.stderr:2024-04-22T06:30:45.143+0000 7f05950a4640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6817/0: (98) Address already in use 2024-04-22T06:30:45.156 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T06:30:45.158+0000 7fc89d821040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-22T06:30:45.199 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T06:30:45.201+0000 7fba14012040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-22T06:30:45.438 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T06:30:45.439+0000 7fc89d821040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-22T06:30:45.485 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T06:30:45.486+0000 7fc89d821040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-22T06:30:45.499 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T06:30:45.500+0000 7fba14012040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-22T06:30:45.536 DEBUG:teuthology.orchestra.run.smithi044:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-22T06:30:45.550 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T06:30:45.551+0000 7fba14012040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-22T06:30:45.630 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.631+0000 7fc31163a640 1 Processor -- start 2024-04-22T06:30:45.630 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.631+0000 7fc31163a640 1 -- start start 2024-04-22T06:30:45.632 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.632+0000 7fc31163a640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc30c14ea70 0x7fc30c157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:45.632 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.632+0000 7fc31163a640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc30c06ca50 0x7fc30c0c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:45.632 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.632+0000 7fc31163a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc30c0c6420 0x7fc30c0c8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:45.632 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.632+0000 7fc31163a640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fc30c14ee50 con 0x7fc30c06ca50 2024-04-22T06:30:45.632 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.632+0000 7fc31163a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fc30c06d000 con 0x7fc30c0c6420 2024-04-22T06:30:45.632 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.632+0000 7fc31163a640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc30c06d2c0 con 0x7fc30c14ea70 2024-04-22T06:30:45.632 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.632+0000 7fc303fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc30c06ca50 0x7fc30c0c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:45.632 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.632+0000 7fc303fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc30c06ca50 0x7fc30c0c5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57508/0 (socket says 172.21.15.44:57508) 2024-04-22T06:30:45.632 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.632+0000 7fc303fff640 1 -- 172.21.15.44:0/2332789848 learned_addr learned my addr 172.21.15.44:0/2332789848 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:30:45.632 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.632+0000 7fc30affd640 1 --2- 172.21.15.44:0/2332789848 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc30c14ea70 0x7fc30c157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:45.632 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.632+0000 7fc30b7fe640 1 --2- 172.21.15.44:0/2332789848 >> v2:172.21.15.70:3300/0 conn(0x7fc30c0c6420 0x7fc30c0c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:45.633 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.632+0000 7fc303fff640 1 -- 172.21.15.44:0/2332789848 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc30c14ea70 msgr2=0x7fc30c157fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:30:45.633 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.632+0000 7fc303fff640 1 --2- 172.21.15.44:0/2332789848 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc30c14ea70 0x7fc30c157fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:45.633 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.632+0000 7fc303fff640 1 -- 172.21.15.44:0/2332789848 >> v2:172.21.15.70:3300/0 conn(0x7fc30c0c6420 msgr2=0x7fc30c0c8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:30:45.633 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.632+0000 7fc303fff640 1 --2- 172.21.15.44:0/2332789848 >> v2:172.21.15.70:3300/0 conn(0x7fc30c0c6420 0x7fc30c0c8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:45.633 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.632+0000 7fc303fff640 1 -- 172.21.15.44:0/2332789848 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc30c06d5a0 con 0x7fc30c06ca50 2024-04-22T06:30:45.633 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.632+0000 7fc30affd640 1 --2- 172.21.15.44:0/2332789848 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc30c14ea70 0x7fc30c157fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:30:45.633 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.632+0000 7fc30b7fe640 1 --2- 172.21.15.44:0/2332789848 >> v2:172.21.15.70:3300/0 conn(0x7fc30c0c6420 0x7fc30c0c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:30:45.633 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.633+0000 7fc303fff640 1 --2- 172.21.15.44:0/2332789848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc30c06ca50 0x7fc30c0c5ee0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc30c0cb4b0 tx=0x7fc2f402e170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9acfd7fc1b9d0ed5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:30:45.633 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.633+0000 7fc30a7fc640 1 -- 172.21.15.44:0/2332789848 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f403b070 con 0x7fc30c06ca50 2024-04-22T06:30:45.633 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.633+0000 7fc30a7fc640 1 -- 172.21.15.44:0/2332789848 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2f4036440 con 0x7fc30c06ca50 2024-04-22T06:30:45.633 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.633+0000 7fc30a7fc640 1 -- 172.21.15.44:0/2332789848 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4036810 con 0x7fc30c06ca50 2024-04-22T06:30:45.636 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.634+0000 7fc31163a640 1 -- 172.21.15.44:0/2332789848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc30c06ca50 msgr2=0x7fc30c0c5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:45.636 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.634+0000 7fc31163a640 1 --2- 172.21.15.44:0/2332789848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc30c06ca50 0x7fc30c0c5ee0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc30c0cb4b0 tx=0x7fc2f402e170 comp rx=0 tx=0).stop 2024-04-22T06:30:45.636 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.634+0000 7fc31163a640 1 -- 172.21.15.44:0/2332789848 shutdown_connections 2024-04-22T06:30:45.636 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.634+0000 7fc31163a640 1 --2- 172.21.15.44:0/2332789848 >> v2:172.21.15.70:3300/0 conn(0x7fc30c0c6420 0x7fc30c0c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:45.636 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.634+0000 7fc31163a640 1 --2- 172.21.15.44:0/2332789848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc30c06ca50 0x7fc30c0c5ee0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:45.636 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.634+0000 7fc31163a640 1 --2- 172.21.15.44:0/2332789848 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc30c14ea70 0x7fc30c157fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:45.636 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.634+0000 7fc31163a640 1 -- 172.21.15.44:0/2332789848 >> 172.21.15.44:0/2332789848 conn(0x7fc30c06ff20 msgr2=0x7fc30c0ca100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:30:45.637 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.634+0000 7fc31163a640 1 -- 172.21.15.44:0/2332789848 shutdown_connections 2024-04-22T06:30:45.637 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.634+0000 7fc31163a640 1 -- 172.21.15.44:0/2332789848 wait complete. 2024-04-22T06:30:45.637 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.635+0000 7fc31163a640 1 Processor -- start 2024-04-22T06:30:45.637 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.635+0000 7fc31163a640 1 -- start start 2024-04-22T06:30:45.637 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.636+0000 7fc31163a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc30c06ca50 0x7fc30c14ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:45.637 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.636+0000 7fc31163a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc30c0c6420 0x7fc30c0cba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:45.637 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.636+0000 7fc31163a640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc30c14ea70 0x7fc30c0cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:45.637 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.636+0000 7fc31163a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc30c06de40 con 0x7fc30c06ca50 2024-04-22T06:30:45.637 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.636+0000 7fc31163a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc30c06db60 con 0x7fc30c0c6420 2024-04-22T06:30:45.637 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.636+0000 7fc31163a640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc30c0c9460 con 0x7fc30c14ea70 2024-04-22T06:30:45.637 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.636+0000 7fc30affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc30c06ca50 0x7fc30c14ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:45.637 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.636+0000 7fc30affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc30c06ca50 0x7fc30c14ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57518/0 (socket says 172.21.15.44:57518) 2024-04-22T06:30:45.637 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.636+0000 7fc30affd640 1 -- 172.21.15.44:0/2529114375 learned_addr learned my addr 172.21.15.44:0/2529114375 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:30:45.637 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.636+0000 7fc30b7fe640 1 --2- 172.21.15.44:0/2529114375 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc30c14ea70 0x7fc30c0cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:45.638 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.636+0000 7fc303fff640 1 --2- 172.21.15.44:0/2529114375 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc30c0c6420 0x7fc30c0cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:45.638 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.636+0000 7fc30affd640 1 -- 172.21.15.44:0/2529114375 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc30c14ea70 msgr2=0x7fc30c0cbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:45.638 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.636+0000 7fc30affd640 1 --2- 172.21.15.44:0/2529114375 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc30c14ea70 0x7fc30c0cbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:45.638 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.636+0000 7fc30affd640 1 -- 172.21.15.44:0/2529114375 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc30c0c6420 msgr2=0x7fc30c0cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:45.638 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.636+0000 7fc30affd640 1 --2- 172.21.15.44:0/2529114375 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc30c0c6420 0x7fc30c0cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:45.638 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.636+0000 7fc30affd640 1 -- 172.21.15.44:0/2529114375 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc30c1f0570 con 0x7fc30c06ca50 2024-04-22T06:30:45.638 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.636+0000 7fc30affd640 1 --2- 172.21.15.44:0/2529114375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc30c06ca50 0x7fc30c14ce00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc00d7d0 tx=0x7fc2fc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:30:45.638 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.637+0000 7fc3037fe640 1 -- 172.21.15.44:0/2529114375 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc004030 con 0x7fc30c06ca50 2024-04-22T06:30:45.638 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.637+0000 7fc3037fe640 1 -- 172.21.15.44:0/2529114375 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2fc0041d0 con 0x7fc30c06ca50 2024-04-22T06:30:45.638 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.637+0000 7fc31163a640 1 -- 172.21.15.44:0/2529114375 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc30c1f0850 con 0x7fc30c06ca50 2024-04-22T06:30:45.638 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.637+0000 7fc3037fe640 1 -- 172.21.15.44:0/2529114375 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc00b6c0 con 0x7fc30c06ca50 2024-04-22T06:30:45.638 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.637+0000 7fc31163a640 1 -- 172.21.15.44:0/2529114375 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc30c0ce270 con 0x7fc30c06ca50 2024-04-22T06:30:45.638 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.637+0000 7fc3037fe640 1 -- 172.21.15.44:0/2529114375 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc2fc00b860 con 0x7fc30c06ca50 2024-04-22T06:30:45.638 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.637+0000 7fc3037fe640 1 -- 172.21.15.44:0/2529114375 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fc2fc003de0 con 0x7fc30c06ca50 2024-04-22T06:30:45.639 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.638+0000 7fc31163a640 1 -- 172.21.15.44:0/2529114375 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc30c04fce0 con 0x7fc30c06ca50 2024-04-22T06:30:45.639 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.640+0000 7fc3037fe640 1 -- 172.21.15.44:0/2529114375 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fc2fc00bc40 con 0x7fc30c06ca50 2024-04-22T06:30:45.639 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T06:30:45.641+0000 7fc89d821040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-22T06:30:45.701 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.703+0000 7fc31163a640 1 -- 172.21.15.44:0/2529114375 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc30c0c9240 con 0x7fc30c06ca50 2024-04-22T06:30:45.702 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.703+0000 7fc3037fe640 1 -- 172.21.15.44:0/2529114375 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fc2fc00be20 con 0x7fc30c06ca50 2024-04-22T06:30:45.702 INFO:teuthology.misc.health.smithi044.stdout: 2024-04-22T06:30:45.702 INFO:teuthology.misc.health.smithi044.stdout:{"epoch":10,"fsid":"9b0a34ca-403d-4513-8a66-d3749ba559aa","created":"2024-04-22T06:30:42.334581+0000","modified":"2024-04-22T06:30:44.467727+0000","last_up_change":"0.000000","last_in_change":"2024-04-22T06:30:44.467727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2b4a6692-53c9-472d-b2f3-1dd29a6679ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"70a6370a-7c0a-4d64-a0e4-b2e726b4f291","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f2e51a62-ca91-470d-bd5a-5710c61deee4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"bc514203-9af9-4f1f-b18e-765bdd5de6e7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"357c439a-6e08-4c19-ac18-b68e6c890497","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"952fb88f-d56e-4305-bfa8-d32674332dfe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"74399b3d-fac2-45ab-9172-2cd95b02450f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"8f35508d-6fcc-43ff-8ee7-0498a7ab3b8d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T06:30:45.704 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.705+0000 7fc31163a640 1 -- 172.21.15.44:0/2529114375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc30c06ca50 msgr2=0x7fc30c14ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:45.704 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.705+0000 7fc31163a640 1 --2- 172.21.15.44:0/2529114375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc30c06ca50 0x7fc30c14ce00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc00d7d0 tx=0x7fc2fc00dc90 comp rx=0 tx=0).stop 2024-04-22T06:30:45.704 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.705+0000 7fc31163a640 1 -- 172.21.15.44:0/2529114375 shutdown_connections 2024-04-22T06:30:45.704 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.705+0000 7fc31163a640 1 --2- 172.21.15.44:0/2529114375 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc30c14ea70 0x7fc30c0cbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:45.704 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.705+0000 7fc31163a640 1 --2- 172.21.15.44:0/2529114375 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc30c0c6420 0x7fc30c0cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:45.704 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.705+0000 7fc31163a640 1 --2- 172.21.15.44:0/2529114375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc30c06ca50 0x7fc30c14ce00 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:45.704 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.705+0000 7fc31163a640 1 -- 172.21.15.44:0/2529114375 >> 172.21.15.44:0/2529114375 conn(0x7fc30c06ff20 msgr2=0x7fc30c0c6fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:30:45.704 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.705+0000 7fc31163a640 1 -- 172.21.15.44:0/2529114375 shutdown_connections 2024-04-22T06:30:45.704 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:45.705+0000 7fc31163a640 1 -- 172.21.15.44:0/2529114375 wait complete. 2024-04-22T06:30:45.717 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-22T06:30:45.726 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T06:30:45.727+0000 7fba14012040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-22T06:30:45.776 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T06:30:45.778+0000 7fc89d821040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-22T06:30:45.824 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T06:30:45.826+0000 7fc89d821040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-22T06:30:45.871 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T06:30:45.873+0000 7fba14012040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-22T06:30:45.893 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T06:30:45.895+0000 7fc89d821040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-22T06:30:45.924 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T06:30:45.925+0000 7fba14012040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-22T06:30:45.938 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T06:30:45.940+0000 7fc89d821040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-22T06:30:45.999 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T06:30:46.000+0000 7fba14012040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-22T06:30:46.047 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T06:30:46.048+0000 7fba14012040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-22T06:30:46.416 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T06:30:46.418+0000 7fc89d821040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-22T06:30:46.532 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T06:30:46.534+0000 7fc89d821040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-22T06:30:46.555 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T06:30:46.556+0000 7fba14012040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-22T06:30:46.678 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T06:30:46.680+0000 7fba14012040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-22T06:30:47.549 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T06:30:47.550+0000 7fc83e181640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-22T06:30:47.549 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T06:30:47.550+0000 7fc83e181640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-22T06:30:48.550 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-04-22T06:30:48.551+0000 7f4c4d814640 -1 osd.2 0 log_to_monitors true 2024-04-22T06:30:48.606 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-22T06:30:48.608+0000 7f0514323640 -1 osd.0 0 log_to_monitors true 2024-04-22T06:30:48.616 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-04-22T06:30:48.618+0000 7fa72b413640 -1 osd.1 0 log_to_monitors true 2024-04-22T06:30:48.636 INFO:tasks.ceph.osd.3.smithi044.stderr:2024-04-22T06:30:48.638+0000 7f255f015640 -1 osd.3 0 log_to_monitors true 2024-04-22T06:30:48.650 INFO:tasks.ceph.osd.4.smithi070.stderr:2024-04-22T06:30:48.652+0000 7f0596625640 -1 osd.4 0 log_to_monitors true 2024-04-22T06:30:48.676 INFO:tasks.ceph.osd.5.smithi070.stderr:2024-04-22T06:30:48.678+0000 7fb039213640 -1 osd.5 0 log_to_monitors true 2024-04-22T06:30:48.679 INFO:tasks.ceph.osd.7.smithi070.stderr:2024-04-22T06:30:48.680+0000 7ff5f601f640 -1 osd.7 0 log_to_monitors true 2024-04-22T06:30:48.679 INFO:tasks.ceph.osd.6.smithi070.stderr:2024-04-22T06:30:48.680+0000 7f1fe47dc640 -1 osd.6 0 log_to_monitors true 2024-04-22T06:30:51.926 INFO:tasks.ceph.osd.7.smithi070.stderr:2024-04-22T06:30:51.926+0000 7ff5f1a12640 -1 osd.7 0 waiting for initial osdmap 2024-04-22T06:30:51.934 INFO:tasks.ceph.osd.7.smithi070.stderr:2024-04-22T06:30:51.936+0000 7ff5e9832640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T06:30:51.952 INFO:tasks.ceph.osd.4.smithi070.stderr:2024-04-22T06:30:51.954+0000 7f059208b640 -1 osd.4 0 waiting for initial osdmap 2024-04-22T06:30:51.965 INFO:tasks.ceph.osd.4.smithi070.stderr:2024-04-22T06:30:51.967+0000 7f0589eab640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T06:30:51.967 INFO:tasks.ceph.osd.5.smithi070.stderr:2024-04-22T06:30:51.968+0000 7fb034bce640 -1 osd.5 0 waiting for initial osdmap 2024-04-22T06:30:51.975 INFO:tasks.ceph.osd.5.smithi070.stderr:2024-04-22T06:30:51.977+0000 7fb02c9ee640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T06:30:51.996 INFO:tasks.ceph.osd.6.smithi070.stderr:2024-04-22T06:30:51.998+0000 7f1fe0f86640 -1 osd.6 0 waiting for initial osdmap 2024-04-22T06:30:52.009 INFO:tasks.ceph.osd.6.smithi070.stderr:2024-04-22T06:30:52.011+0000 7f1fd8592640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T06:30:52.524 DEBUG:teuthology.orchestra.run.smithi044:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-22T06:30:52.560 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-04-22T06:30:52.560+0000 7fa726dff640 -1 osd.1 0 waiting for initial osdmap 2024-04-22T06:30:52.589 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-04-22T06:30:52.591+0000 7fa726dff640 -1 osd.1 0 waiting for initial osdmap 2024-04-22T06:30:52.591 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-04-22T06:30:52.592+0000 7fa71ec1f640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T06:30:52.627 INFO:tasks.ceph.osd.3.smithi044.stderr:2024-04-22T06:30:52.628+0000 7f255a9b2640 -1 osd.3 0 waiting for initial osdmap 2024-04-22T06:30:52.642 INFO:tasks.ceph.osd.3.smithi044.stderr:2024-04-22T06:30:52.644+0000 7f25527d2640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T06:30:52.647 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-22T06:30:52.649+0000 7f05102b9640 -1 osd.0 0 waiting for initial osdmap 2024-04-22T06:30:52.660 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.661+0000 7f489ba55640 1 Processor -- start 2024-04-22T06:30:52.660 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.662+0000 7f489ba55640 1 -- start start 2024-04-22T06:30:52.661 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-22T06:30:52.663+0000 7f05102b9640 -1 osd.0 0 waiting for initial osdmap 2024-04-22T06:30:52.662 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.663+0000 7f489ba55640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f48940c87a0 0x7f4894153240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:52.662 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.663+0000 7f489ba55640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48940cb6e0 con 0x7f48940ced30 2024-04-22T06:30:52.662 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.663+0000 7f489ba55640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48940cb9c0 con 0x7f48940ce950 2024-04-22T06:30:52.663 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.663+0000 7f4898ff0640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f48940ced30 0x7f4894152b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:47760/0 (socket says 172.21.15.44:47760) 2024-04-22T06:30:52.663 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.663+0000 7f489ba55640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f48940ca1e0 con 0x7f48940c87a0 2024-04-22T06:30:52.663 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.663+0000 7f4898ff0640 1 -- 172.21.15.44:0/1028386719 learned_addr learned my addr 172.21.15.44:0/1028386719 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:30:52.663 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.663+0000 7f48997f1640 1 --1- 172.21.15.44:0/1028386719 >> v1:172.21.15.70:6789/0 conn(0x7f48940ce950 0x7f48940c8090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.44:33880/0 (socket says 172.21.15.44:33880) 2024-04-22T06:30:52.663 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.663+0000 7f4899ff2640 1 --2- 172.21.15.44:0/1028386719 >> v2:172.21.15.70:3300/0 conn(0x7f48940c87a0 0x7f4894153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:52.663 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.663+0000 7f488bfff640 1 -- 172.21.15.44:0/1028386719 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2759936830 0 0) 0x7f48940cb9c0 con 0x7f48940ce950 2024-04-22T06:30:52.663 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.663+0000 7f488bfff640 1 -- 172.21.15.44:0/1028386719 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4880003520 con 0x7f48940ce950 2024-04-22T06:30:52.663 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.663+0000 7f488bfff640 1 -- 172.21.15.44:0/1028386719 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2407795185 0 0) 0x7f48940cb6e0 con 0x7f48940ced30 2024-04-22T06:30:52.663 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.663+0000 7f488bfff640 1 -- 172.21.15.44:0/1028386719 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f48940cb9c0 con 0x7f48940ced30 2024-04-22T06:30:52.663 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.663+0000 7f4899ff2640 1 -- 172.21.15.44:0/1028386719 >> v1:172.21.15.70:6789/0 conn(0x7f48940ce950 legacy=0x7f48940c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:52.663 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.663+0000 7f4899ff2640 1 -- 172.21.15.44:0/1028386719 >> v1:172.21.15.44:6789/0 conn(0x7f48940ced30 legacy=0x7f4894152b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:52.663 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.663+0000 7f4899ff2640 1 -- 172.21.15.44:0/1028386719 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48940ca430 con 0x7f48940c87a0 2024-04-22T06:30:52.664 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-22T06:30:52.664+0000 7f05080d9640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T06:30:52.665 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.664+0000 7f4899ff2640 1 --2- 172.21.15.44:0/1028386719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48940c87a0 0x7f4894153240 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f48940ca360 tx=0x7f488c02bf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dd7003225414cf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:30:52.665 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.664+0000 7f488bfff640 1 -- 172.21.15.44:0/1028386719 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c039070 con 0x7f48940c87a0 2024-04-22T06:30:52.665 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.664+0000 7f488bfff640 1 -- 172.21.15.44:0/1028386719 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f488c02ccd0 con 0x7f48940c87a0 2024-04-22T06:30:52.665 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.664+0000 7f488bfff640 1 -- 172.21.15.44:0/1028386719 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c034640 con 0x7f48940c87a0 2024-04-22T06:30:52.665 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.665+0000 7f489ba55640 1 -- 172.21.15.44:0/1028386719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48940c87a0 msgr2=0x7f4894153240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:52.665 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.665+0000 7f489ba55640 1 --2- 172.21.15.44:0/1028386719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48940c87a0 0x7f4894153240 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f48940ca360 tx=0x7f488c02bf20 comp rx=0 tx=0).stop 2024-04-22T06:30:52.665 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.665+0000 7f489ba55640 1 -- 172.21.15.44:0/1028386719 shutdown_connections 2024-04-22T06:30:52.665 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.665+0000 7f489ba55640 1 --2- 172.21.15.44:0/1028386719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48940c87a0 0x7f4894153240 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:52.665 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.665+0000 7f489ba55640 1 -- 172.21.15.44:0/1028386719 >> 172.21.15.44:0/1028386719 conn(0x7f489406bd60 msgr2=0x7f48940707e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:30:52.665 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.666+0000 7f489ba55640 1 -- 172.21.15.44:0/1028386719 shutdown_connections 2024-04-22T06:30:52.665 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.666+0000 7f489ba55640 1 -- 172.21.15.44:0/1028386719 wait complete. 2024-04-22T06:30:52.668 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.667+0000 7f489ba55640 1 Processor -- start 2024-04-22T06:30:52.668 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.667+0000 7f489ba55640 1 -- start start 2024-04-22T06:30:52.668 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.667+0000 7f489ba55640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48940c87a0 0x7f48941f4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:52.668 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.668+0000 7f489ba55640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f48940ce950 0x7f4894147da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:52.668 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.668+0000 7f489ba55640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48940ced30 0x7f4894144390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:52.668 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.668+0000 7f489ba55640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f48940caeb0 con 0x7f48940c87a0 2024-04-22T06:30:52.668 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.668+0000 7f489ba55640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f48940cad30 con 0x7f48940ced30 2024-04-22T06:30:52.668 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.668+0000 7f489ba55640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f48940cbe00 con 0x7f48940ce950 2024-04-22T06:30:52.668 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.668+0000 7f48997f1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48940c87a0 0x7f48941f4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:52.668 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.668+0000 7f4899ff2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48940ced30 0x7f4894144390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:52.668 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.668+0000 7f4899ff2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48940ced30 0x7f4894144390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:47606/0 (socket says 172.21.15.44:47606) 2024-04-22T06:30:52.668 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.668+0000 7f4899ff2640 1 -- 172.21.15.44:0/2030064361 learned_addr learned my addr 172.21.15.44:0/2030064361 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:30:52.668 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.668+0000 7f4898ff0640 1 --2- 172.21.15.44:0/2030064361 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f48940ce950 0x7f4894147da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:52.669 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.668+0000 7f4899ff2640 1 -- 172.21.15.44:0/2030064361 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f48940ce950 msgr2=0x7f4894147da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:52.669 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.668+0000 7f4899ff2640 1 --2- 172.21.15.44:0/2030064361 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f48940ce950 0x7f4894147da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:52.669 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.668+0000 7f4899ff2640 1 -- 172.21.15.44:0/2030064361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48940c87a0 msgr2=0x7f48941f4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:52.669 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.668+0000 7f4899ff2640 1 --2- 172.21.15.44:0/2030064361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48940c87a0 0x7f48941f4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:52.669 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.668+0000 7f4899ff2640 1 -- 172.21.15.44:0/2030064361 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48941f09e0 con 0x7f48940ced30 2024-04-22T06:30:52.669 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.668+0000 7f4899ff2640 1 --2- 172.21.15.44:0/2030064361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48940ced30 0x7f4894144390 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f488c02c530 tx=0x7f488c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:30:52.669 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.669+0000 7f4889ffb640 1 -- 172.21.15.44:0/2030064361 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c039050 con 0x7f48940ced30 2024-04-22T06:30:52.669 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.669+0000 7f4889ffb640 1 -- 172.21.15.44:0/2030064361 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f488c045070 con 0x7f48940ced30 2024-04-22T06:30:52.669 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.669+0000 7f4889ffb640 1 -- 172.21.15.44:0/2030064361 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c009450 con 0x7f48940ced30 2024-04-22T06:30:52.669 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.669+0000 7f489ba55640 1 -- 172.21.15.44:0/2030064361 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48941f0ca0 con 0x7f48940ced30 2024-04-22T06:30:52.669 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.669+0000 7f489ba55640 1 -- 172.21.15.44:0/2030064361 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48941f1260 con 0x7f48940ced30 2024-04-22T06:30:52.670 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.670+0000 7f489ba55640 1 -- 172.21.15.44:0/2030064361 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4894075e40 con 0x7f48940ced30 2024-04-22T06:30:52.670 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.670+0000 7f4889ffb640 1 -- 172.21.15.44:0/2030064361 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f488c03e830 con 0x7f48940ced30 2024-04-22T06:30:52.670 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.671+0000 7f4889ffb640 1 --2- 172.21.15.44:0/2030064361 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f487806c860 0x7f487806ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:52.671 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.671+0000 7f4889ffb640 1 -- 172.21.15.44:0/2030064361 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f488c0ad8d0 con 0x7f48940ced30 2024-04-22T06:30:52.671 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.671+0000 7f48997f1640 1 --2- 172.21.15.44:0/2030064361 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f487806c860 0x7f487806ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:52.671 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.671+0000 7f48997f1640 1 --2- 172.21.15.44:0/2030064361 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f487806c860 0x7f487806ed20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4890000f60 tx=0x7f4890007d50 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:30:52.674 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.675+0000 7f4889ffb640 1 -- 172.21.15.44:0/2030064361 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4894075e40 con 0x7f48940ced30 2024-04-22T06:30:52.808 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-04-22T06:30:52.809+0000 7f4c4927d640 -1 osd.2 0 waiting for initial osdmap 2024-04-22T06:30:52.822 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-04-22T06:30:52.824+0000 7f4c4109d640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T06:30:52.869 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.870+0000 7f489ba55640 1 -- 172.21.15.44:0/2030064361 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f489404fce0 con 0x7f48940ced30 2024-04-22T06:30:52.870 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.871+0000 7f4889ffb640 1 -- 172.21.15.44:0/2030064361 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f488c07c790 con 0x7f48940ced30 2024-04-22T06:30:52.870 INFO:teuthology.misc.health.smithi044.stdout: 2024-04-22T06:30:52.870 INFO:teuthology.misc.health.smithi044.stdout:{"epoch":12,"fsid":"9b0a34ca-403d-4513-8a66-d3749ba559aa","created":"2024-04-22T06:30:42.334581+0000","modified":"2024-04-22T06:30:50.548373+0000","last_up_change":"0.000000","last_in_change":"2024-04-22T06:30:44.467727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2b4a6692-53c9-472d-b2f3-1dd29a6679ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"70a6370a-7c0a-4d64-a0e4-b2e726b4f291","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f2e51a62-ca91-470d-bd5a-5710c61deee4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"bc514203-9af9-4f1f-b18e-765bdd5de6e7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"357c439a-6e08-4c19-ac18-b68e6c890497","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"952fb88f-d56e-4305-bfa8-d32674332dfe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"74399b3d-fac2-45ab-9172-2cd95b02450f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"8f35508d-6fcc-43ff-8ee7-0498a7ab3b8d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T06:30:52.872 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.873+0000 7f489ba55640 1 -- 172.21.15.44:0/2030064361 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f487806c860 msgr2=0x7f487806ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:52.872 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.873+0000 7f489ba55640 1 --2- 172.21.15.44:0/2030064361 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f487806c860 0x7f487806ed20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4890000f60 tx=0x7f4890007d50 comp rx=0 tx=0).stop 2024-04-22T06:30:52.872 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.873+0000 7f489ba55640 1 -- 172.21.15.44:0/2030064361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48940ced30 msgr2=0x7f4894144390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:52.872 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.873+0000 7f489ba55640 1 --2- 172.21.15.44:0/2030064361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48940ced30 0x7f4894144390 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f488c02c530 tx=0x7f488c004a40 comp rx=0 tx=0).stop 2024-04-22T06:30:52.872 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.873+0000 7f489ba55640 1 -- 172.21.15.44:0/2030064361 shutdown_connections 2024-04-22T06:30:52.872 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.873+0000 7f489ba55640 1 --2- 172.21.15.44:0/2030064361 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f487806c860 0x7f487806ed20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:52.873 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.873+0000 7f489ba55640 1 --2- 172.21.15.44:0/2030064361 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f48940ce950 0x7f4894147da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:52.873 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.873+0000 7f489ba55640 1 --2- 172.21.15.44:0/2030064361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48940ced30 0x7f4894144390 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:52.873 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.873+0000 7f489ba55640 1 --2- 172.21.15.44:0/2030064361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48940c87a0 0x7f48941f4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:52.873 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.873+0000 7f489ba55640 1 -- 172.21.15.44:0/2030064361 >> 172.21.15.44:0/2030064361 conn(0x7f489406bd60 msgr2=0x7f4894067800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:30:52.873 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.873+0000 7f489ba55640 1 -- 172.21.15.44:0/2030064361 shutdown_connections 2024-04-22T06:30:52.873 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:52.873+0000 7f489ba55640 1 -- 172.21.15.44:0/2030064361 wait complete. 2024-04-22T06:30:52.881 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-22T06:30:59.688 DEBUG:teuthology.orchestra.run.smithi044:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-22T06:30:59.792 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.793+0000 7f6103f15640 1 Processor -- start 2024-04-22T06:30:59.792 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.793+0000 7f6103f15640 1 -- start start 2024-04-22T06:30:59.794 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.794+0000 7f6103f15640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f60fc073800 0x7f60fc15b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:59.794 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.794+0000 7f6103f15640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60fc06d490 con 0x7f60fc073420 2024-04-22T06:30:59.794 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.794+0000 7f6103f15640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60fc15e540 con 0x7f60fc15c440 2024-04-22T06:30:59.794 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.794+0000 7f6103f15640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f60fc15f150 con 0x7f60fc073800 2024-04-22T06:30:59.794 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.794+0000 7f6101cb1640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f60fc073420 0x7f60fc06ca50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:47776/0 (socket says 172.21.15.44:47776) 2024-04-22T06:30:59.794 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.794+0000 7f6101cb1640 1 -- 172.21.15.44:0/1944302776 learned_addr learned my addr 172.21.15.44:0/1944302776 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:30:59.794 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.794+0000 7f61014b0640 1 --2- 172.21.15.44:0/1944302776 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f60fc073800 0x7f60fc15b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:59.794 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.795+0000 7f6100caf640 1 -- 172.21.15.44:0/1944302776 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3577951796 0 0) 0x7f60fc06d490 con 0x7f60fc073420 2024-04-22T06:30:59.794 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.795+0000 7f6100caf640 1 -- 172.21.15.44:0/1944302776 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60e4003520 con 0x7f60fc073420 2024-04-22T06:30:59.795 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.795+0000 7f6100caf640 1 -- 172.21.15.44:0/1944302776 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2026012830 0 0) 0x7f60fc15e540 con 0x7f60fc15c440 2024-04-22T06:30:59.795 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.795+0000 7f6100caf640 1 -- 172.21.15.44:0/1944302776 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60fc06d490 con 0x7f60fc15c440 2024-04-22T06:30:59.795 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.795+0000 7f61014b0640 1 -- 172.21.15.44:0/1944302776 >> v1:172.21.15.70:6789/0 conn(0x7f60fc15c440 legacy=0x7f60fc154d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:59.795 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.795+0000 7f61014b0640 1 -- 172.21.15.44:0/1944302776 >> v1:172.21.15.44:6789/0 conn(0x7f60fc073420 legacy=0x7f60fc06ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:59.795 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.795+0000 7f6100caf640 1 -- 172.21.15.44:0/1944302776 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2698040963 0 0) 0x7f60e4003520 con 0x7f60fc073420 2024-04-22T06:30:59.795 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.795+0000 7f61014b0640 1 -- 172.21.15.44:0/1944302776 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60fc15e820 con 0x7f60fc073800 2024-04-22T06:30:59.796 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.796+0000 7f61014b0640 1 --2- 172.21.15.44:0/1944302776 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f60fc073800 0x7f60fc15b980 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f60f0009800 tx=0x7f60f002b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38cfa8582f2cb0a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:30:59.796 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.796+0000 7f6100caf640 1 -- 172.21.15.44:0/1944302776 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f4001670 con 0x7f60fc073800 2024-04-22T06:30:59.797 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.796+0000 7f6100caf640 1 -- 172.21.15.44:0/1944302776 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60f0037070 con 0x7f60fc073800 2024-04-22T06:30:59.797 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.796+0000 7f6100caf640 1 -- 172.21.15.44:0/1944302776 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f0031760 con 0x7f60fc073800 2024-04-22T06:30:59.797 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.796+0000 7f6103f15640 1 -- 172.21.15.44:0/1944302776 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f60fc073800 msgr2=0x7f60fc15b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:59.797 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.796+0000 7f6103f15640 1 --2- 172.21.15.44:0/1944302776 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f60fc073800 0x7f60fc15b980 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f60f0009800 tx=0x7f60f002b880 comp rx=0 tx=0).stop 2024-04-22T06:30:59.797 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.796+0000 7f6103f15640 1 -- 172.21.15.44:0/1944302776 shutdown_connections 2024-04-22T06:30:59.797 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.796+0000 7f6103f15640 1 --2- 172.21.15.44:0/1944302776 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f60fc073800 0x7f60fc15b980 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:59.797 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.796+0000 7f6103f15640 1 -- 172.21.15.44:0/1944302776 >> 172.21.15.44:0/1944302776 conn(0x7f60fc06bd60 msgr2=0x7f60fc0708a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:30:59.797 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.797+0000 7f6103f15640 1 -- 172.21.15.44:0/1944302776 shutdown_connections 2024-04-22T06:30:59.797 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.797+0000 7f6103f15640 1 -- 172.21.15.44:0/1944302776 wait complete. 2024-04-22T06:30:59.797 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.798+0000 7f6103f15640 1 Processor -- start 2024-04-22T06:30:59.797 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.798+0000 7f6103f15640 1 -- start start 2024-04-22T06:30:59.800 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.798+0000 7f6103f15640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60fc073420 0x7f60fc1e6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:59.800 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.798+0000 7f6103f15640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60fc073800 0x7f60fc1f1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:59.800 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.798+0000 7f6103f15640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f60fc15c440 0x7f60fc0cace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:59.800 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.798+0000 7f6103f15640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc06d160 con 0x7f60fc073420 2024-04-22T06:30:59.800 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.798+0000 7f6103f15640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc15ede0 con 0x7f60fc073800 2024-04-22T06:30:59.800 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.798+0000 7f6103f15640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f60fc155450 con 0x7f60fc15c440 2024-04-22T06:30:59.800 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.798+0000 7f6101cb1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60fc073420 0x7f60fc1e6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:59.800 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.799+0000 7f6101cb1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60fc073420 0x7f60fc1e6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54734/0 (socket says 172.21.15.44:54734) 2024-04-22T06:30:59.800 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.799+0000 7f6101cb1640 1 -- 172.21.15.44:0/4183565082 learned_addr learned my addr 172.21.15.44:0/4183565082 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:30:59.801 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.799+0000 7f61014b0640 1 --2- 172.21.15.44:0/4183565082 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60fc073800 0x7f60fc1f1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:59.801 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.799+0000 7f61024b2640 1 --2- 172.21.15.44:0/4183565082 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f60fc15c440 0x7f60fc0cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:59.801 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.799+0000 7f6101cb1640 1 -- 172.21.15.44:0/4183565082 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f60fc15c440 msgr2=0x7f60fc0cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:59.801 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.799+0000 7f6101cb1640 1 --2- 172.21.15.44:0/4183565082 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f60fc15c440 0x7f60fc0cace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:59.801 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.799+0000 7f6101cb1640 1 -- 172.21.15.44:0/4183565082 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60fc073800 msgr2=0x7f60fc1f1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:59.801 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.799+0000 7f6101cb1640 1 --2- 172.21.15.44:0/4183565082 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60fc073800 0x7f60fc1f1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:59.801 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.799+0000 7f6101cb1640 1 -- 172.21.15.44:0/4183565082 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60fc1e5770 con 0x7f60fc073420 2024-04-22T06:30:59.801 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.799+0000 7f61014b0640 1 --2- 172.21.15.44:0/4183565082 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60fc073800 0x7f60fc1f1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:30:59.801 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.799+0000 7f6101cb1640 1 --2- 172.21.15.44:0/4183565082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60fc073420 0x7f60fc1e6210 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f60f400c780 tx=0x7f60f400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:30:59.801 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.799+0000 7f60fa7fc640 1 -- 172.21.15.44:0/4183565082 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f4017070 con 0x7f60fc073420 2024-04-22T06:30:59.801 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.799+0000 7f60fa7fc640 1 -- 172.21.15.44:0/4183565082 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60f4005af0 con 0x7f60fc073420 2024-04-22T06:30:59.801 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.799+0000 7f6103f15640 1 -- 172.21.15.44:0/4183565082 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60fc1e6b20 con 0x7f60fc073420 2024-04-22T06:30:59.801 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.799+0000 7f60fa7fc640 1 -- 172.21.15.44:0/4183565082 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f4007070 con 0x7f60fc073420 2024-04-22T06:30:59.802 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.799+0000 7f6103f15640 1 -- 172.21.15.44:0/4183565082 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60fc1e70e0 con 0x7f60fc073420 2024-04-22T06:30:59.802 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.800+0000 7f6103f15640 1 -- 172.21.15.44:0/4183565082 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60fc0c8e00 con 0x7f60fc073420 2024-04-22T06:30:59.802 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.801+0000 7f60fa7fc640 1 -- 172.21.15.44:0/4183565082 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f60f401a070 con 0x7f60fc073420 2024-04-22T06:30:59.802 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.801+0000 7f60fa7fc640 1 --2- 172.21.15.44:0/4183565082 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f60e806c8f0 0x7f60e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:30:59.802 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.801+0000 7f60fa7fc640 1 -- 172.21.15.44:0/4183565082 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f60f408ae00 con 0x7f60fc073420 2024-04-22T06:30:59.802 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.801+0000 7f61014b0640 1 --2- 172.21.15.44:0/4183565082 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f60e806c8f0 0x7f60e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:30:59.802 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.802+0000 7f61014b0640 1 --2- 172.21.15.44:0/4183565082 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f60e806c8f0 0x7f60e806edb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f60f0036000 tx=0x7f60f003d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:30:59.805 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.806+0000 7f60fa7fc640 1 -- 172.21.15.44:0/4183565082 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f60fc0c8e00 con 0x7f60fc073420 2024-04-22T06:30:59.994 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.995+0000 7f6103f15640 1 -- 172.21.15.44:0/4183565082 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f60fc04fce0 con 0x7f60fc073420 2024-04-22T06:30:59.995 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.996+0000 7f60fa7fc640 1 -- 172.21.15.44:0/4183565082 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12836 (secure 0 0 0) 0x7f60f4059580 con 0x7f60fc073420 2024-04-22T06:30:59.995 INFO:teuthology.misc.health.smithi044.stdout: 2024-04-22T06:30:59.995 INFO:teuthology.misc.health.smithi044.stdout:{"epoch":17,"fsid":"9b0a34ca-403d-4513-8a66-d3749ba559aa","created":"2024-04-22T06:30:42.334581+0000","modified":"2024-04-22T06:30:56.957495+0000","last_up_change":"2024-04-22T06:30:52.942257+0000","last_in_change":"2024-04-22T06:30:44.467727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T06:30:54.560256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2b4a6692-53c9-472d-b2f3-1dd29a6679ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6817","nonce":2220268493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6819","nonce":2220268493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6823","nonce":2220268493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6821","nonce":2220268493}]},"public_addr":"172.21.15.44:6817/2220268493","cluster_addr":"172.21.15.44:6819/2220268493","heartbeat_back_addr":"172.21.15.44:6823/2220268493","heartbeat_front_addr":"172.21.15.44:6821/2220268493","state":["exists","up"]},{"osd":1,"uuid":"70a6370a-7c0a-4d64-a0e4-b2e726b4f291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6809","nonce":1278678977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6811","nonce":1278678977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6815","nonce":1278678977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6813","nonce":1278678977}]},"public_addr":"172.21.15.44:6809/1278678977","cluster_addr":"172.21.15.44:6811/1278678977","heartbeat_back_addr":"172.21.15.44:6815/1278678977","heartbeat_front_addr":"172.21.15.44:6813/1278678977","state":["exists","up"]},{"osd":2,"uuid":"f2e51a62-ca91-470d-bd5a-5710c61deee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6801","nonce":2544087903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6803","nonce":2544087903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6807","nonce":2544087903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6805","nonce":2544087903}]},"public_addr":"172.21.15.44:6801/2544087903","cluster_addr":"172.21.15.44:6803/2544087903","heartbeat_back_addr":"172.21.15.44:6807/2544087903","heartbeat_front_addr":"172.21.15.44:6805/2544087903","state":["exists","up"]},{"osd":3,"uuid":"bc514203-9af9-4f1f-b18e-765bdd5de6e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6825","nonce":1692928109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6827","nonce":1692928109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6831","nonce":1692928109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6829","nonce":1692928109}]},"public_addr":"172.21.15.44:6825/1692928109","cluster_addr":"172.21.15.44:6827/1692928109","heartbeat_back_addr":"172.21.15.44:6831/1692928109","heartbeat_front_addr":"172.21.15.44:6829/1692928109","state":["exists","up"]},{"osd":4,"uuid":"357c439a-6e08-4c19-ac18-b68e6c890497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6820","nonce":3802577427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6824","nonce":3802577427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6831","nonce":3802577427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6828","nonce":3802577427}]},"public_addr":"172.21.15.70:6820/3802577427","cluster_addr":"172.21.15.70:6824/3802577427","heartbeat_back_addr":"172.21.15.70:6831/3802577427","heartbeat_front_addr":"172.21.15.70:6828/3802577427","state":["exists","up"]},{"osd":5,"uuid":"952fb88f-d56e-4305-bfa8-d32674332dfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6801","nonce":664757345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6803","nonce":664757345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6807","nonce":664757345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6805","nonce":664757345}]},"public_addr":"172.21.15.70:6801/664757345","cluster_addr":"172.21.15.70:6803/664757345","heartbeat_back_addr":"172.21.15.70:6807/664757345","heartbeat_front_addr":"172.21.15.70:6805/664757345","state":["exists","up"]},{"osd":6,"uuid":"74399b3d-fac2-45ab-9172-2cd95b02450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6817","nonce":1623253482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6821","nonce":1623253482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6827","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6829","nonce":1623253482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6825","nonce":1623253482}]},"public_addr":"172.21.15.70:6817/1623253482","cluster_addr":"172.21.15.70:6821/1623253482","heartbeat_back_addr":"172.21.15.70:6829/1623253482","heartbeat_front_addr":"172.21.15.70:6825/1623253482","state":["exists","up"]},{"osd":7,"uuid":"8f35508d-6fcc-43ff-8ee7-0498a7ab3b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6809","nonce":1551180300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6811","nonce":1551180300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6815","nonce":1551180300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6813","nonce":1551180300}]},"public_addr":"172.21.15.70:6809/1551180300","cluster_addr":"172.21.15.70:6811/1551180300","heartbeat_back_addr":"172.21.15.70:6815/1551180300","heartbeat_front_addr":"172.21.15.70:6813/1551180300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.612502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.653756+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.521169+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.649955+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.640231+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.686638+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.642199+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.700360+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T06:30:59.997 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.998+0000 7f6103f15640 1 -- 172.21.15.44:0/4183565082 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f60e806c8f0 msgr2=0x7f60e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:59.997 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.998+0000 7f6103f15640 1 --2- 172.21.15.44:0/4183565082 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f60e806c8f0 0x7f60e806edb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f60f0036000 tx=0x7f60f003d040 comp rx=0 tx=0).stop 2024-04-22T06:30:59.997 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.998+0000 7f6103f15640 1 -- 172.21.15.44:0/4183565082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60fc073420 msgr2=0x7f60fc1e6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:30:59.997 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.998+0000 7f6103f15640 1 --2- 172.21.15.44:0/4183565082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60fc073420 0x7f60fc1e6210 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f60f400c780 tx=0x7f60f400cc40 comp rx=0 tx=0).stop 2024-04-22T06:30:59.998 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.998+0000 7f6103f15640 1 -- 172.21.15.44:0/4183565082 shutdown_connections 2024-04-22T06:30:59.998 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.998+0000 7f6103f15640 1 --2- 172.21.15.44:0/4183565082 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f60e806c8f0 0x7f60e806edb0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:59.998 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.998+0000 7f6103f15640 1 --2- 172.21.15.44:0/4183565082 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f60fc15c440 0x7f60fc0cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:59.998 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.998+0000 7f6103f15640 1 --2- 172.21.15.44:0/4183565082 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60fc073800 0x7f60fc1f1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:59.998 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.998+0000 7f6103f15640 1 --2- 172.21.15.44:0/4183565082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60fc073420 0x7f60fc1e6210 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:30:59.998 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.998+0000 7f6103f15640 1 -- 172.21.15.44:0/4183565082 >> 172.21.15.44:0/4183565082 conn(0x7f60fc06bd60 msgr2=0x7f60fc1e7400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:30:59.998 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.998+0000 7f6103f15640 1 -- 172.21.15.44:0/4183565082 shutdown_connections 2024-04-22T06:30:59.998 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T06:30:59.998+0000 7f6103f15640 1 -- 172.21.15.44:0/4183565082 wait complete. 2024-04-22T06:31:00.007 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-22T06:31:00.007 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-22T06:31:00.007 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-22T06:31:00.007 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-22T06:31:00.010 INFO:tasks.ceph.mds.a:Started 2024-04-22T06:31:00.010 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-22T06:31:00.010 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-22T06:31:00.012 INFO:tasks.ceph.mds.c:Started 2024-04-22T06:31:00.012 INFO:tasks.ceph.mds.e:Restarting daemon 2024-04-22T06:31:00.012 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-04-22T06:31:00.015 INFO:tasks.ceph.mds.e:Started 2024-04-22T06:31:00.015 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-22T06:31:00.015 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-22T06:31:00.017 INFO:tasks.ceph.mds.b:Started 2024-04-22T06:31:00.018 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-22T06:31:00.018 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-22T06:31:00.020 INFO:tasks.ceph.mds.d:Started 2024-04-22T06:31:00.020 INFO:tasks.ceph.mds.f:Restarting daemon 2024-04-22T06:31:00.020 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2024-04-22T06:31:00.022 INFO:tasks.ceph.mds.f:Started 2024-04-22T06:31:00.022 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-22T06:31:00.023 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T06:31:00.087 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-04-22T06:31:00.088 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-04-22T06:31:00.094 INFO:tasks.ceph.mds.e.smithi044.stdout:starting mds.e at 2024-04-22T06:31:00.103 INFO:tasks.ceph.mds.b.smithi070.stdout:starting mds.b at 2024-04-22T06:31:00.104 INFO:tasks.ceph.mds.f.smithi070.stdout:starting mds.f at 2024-04-22T06:31:00.104 INFO:tasks.ceph.mds.d.smithi070.stdout:starting mds.d at 2024-04-22T06:31:00.104 INFO:tasks.ceph.mds.b.smithi070.stderr:2024-04-22T06:31:00.105+0000 7f83953aa640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6837/0: (98) Address already in use 2024-04-22T06:31:00.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.133+0000 7f87b29ab640 1 Processor -- start 2024-04-22T06:31:00.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.133+0000 7f87b29ab640 1 -- start start 2024-04-22T06:31:00.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.134+0000 7f87b29ab640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87ac154e70 0x7f87ac0cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:00.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.134+0000 7f87b29ab640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f87ac0ca7f0 0x7f87ac15e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:00.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.134+0000 7f87b29ab640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87ac0ce520 con 0x7f87ac0ce070 2024-04-22T06:31:00.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.134+0000 7f87b29ab640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f87ac15b320 con 0x7f87ac0ca7f0 2024-04-22T06:31:00.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.134+0000 7f87b29ab640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f87ac15b600 con 0x7f87ac154e70 2024-04-22T06:31:00.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.135+0000 7f87abfff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87ac154e70 0x7f87ac0cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:00.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.135+0000 7f87ab7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f87ac0ce070 0x7f87ac0ca0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.44:33896/0 (socket says 172.21.15.44:33896) 2024-04-22T06:31:00.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.135+0000 7f87ab7fe640 1 -- 172.21.15.44:0/3667591502 learned_addr learned my addr 172.21.15.44:0/3667591502 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:00.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.135+0000 7f87b0f48640 1 --2- 172.21.15.44:0/3667591502 >> v2:172.21.15.70:3300/0 conn(0x7f87ac0ca7f0 0x7f87ac15e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:00.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.135+0000 7f87abfff640 1 -- 172.21.15.44:0/3667591502 >> v2:172.21.15.70:3300/0 conn(0x7f87ac0ca7f0 msgr2=0x7f87ac15e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:00.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.135+0000 7f87abfff640 1 --2- 172.21.15.44:0/3667591502 >> v2:172.21.15.70:3300/0 conn(0x7f87ac0ca7f0 0x7f87ac15e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:00.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.135+0000 7f87abfff640 1 -- 172.21.15.44:0/3667591502 >> v1:172.21.15.70:6789/0 conn(0x7f87ac0ce070 legacy=0x7f87ac0ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:00.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.135+0000 7f87abfff640 1 -- 172.21.15.44:0/3667591502 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87ac15b8e0 con 0x7f87ac154e70 2024-04-22T06:31:00.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.135+0000 7f87b0f48640 1 --2- 172.21.15.44:0/3667591502 >> v2:172.21.15.70:3300/0 conn(0x7f87ac0ca7f0 0x7f87ac15e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:31:00.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.136+0000 7f87abfff640 1 --2- 172.21.15.44:0/3667591502 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87ac154e70 0x7f87ac0cdb30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f87940097a0 tx=0x7f879402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94495a626676eb9f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:00.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.136+0000 7f87aaffd640 1 -- 172.21.15.44:0/3667591502 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87940315a0 con 0x7f87ac154e70 2024-04-22T06:31:00.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.136+0000 7f87aaffd640 1 -- 172.21.15.44:0/3667591502 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8794037070 con 0x7f87ac154e70 2024-04-22T06:31:00.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.136+0000 7f87aaffd640 1 -- 172.21.15.44:0/3667591502 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8794031960 con 0x7f87ac154e70 2024-04-22T06:31:00.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.136+0000 7f87b29ab640 1 -- 172.21.15.44:0/3667591502 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87ac154e70 msgr2=0x7f87ac0cdb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:00.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.136+0000 7f87b29ab640 1 --2- 172.21.15.44:0/3667591502 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87ac154e70 0x7f87ac0cdb30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f87940097a0 tx=0x7f879402b840 comp rx=0 tx=0).stop 2024-04-22T06:31:00.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.136+0000 7f87b29ab640 1 -- 172.21.15.44:0/3667591502 shutdown_connections 2024-04-22T06:31:00.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.136+0000 7f87b29ab640 1 --2- 172.21.15.44:0/3667591502 >> v2:172.21.15.70:3300/0 conn(0x7f87ac0ca7f0 0x7f87ac15e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:00.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.136+0000 7f87b29ab640 1 --2- 172.21.15.44:0/3667591502 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87ac154e70 0x7f87ac0cdb30 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:00.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.136+0000 7f87b29ab640 1 -- 172.21.15.44:0/3667591502 >> 172.21.15.44:0/3667591502 conn(0x7f87ac067f80 msgr2=0x7f87ac06fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:00.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.136+0000 7f87b29ab640 1 -- 172.21.15.44:0/3667591502 shutdown_connections 2024-04-22T06:31:00.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.136+0000 7f87b29ab640 1 -- 172.21.15.44:0/3667591502 wait complete. 2024-04-22T06:31:00.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.137+0000 7f87b29ab640 1 Processor -- start 2024-04-22T06:31:00.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.138+0000 7f87b29ab640 1 -- start start 2024-04-22T06:31:00.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.138+0000 7f87b29ab640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87ac0ce070 0x7f87ac1e7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:00.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.138+0000 7f87b29ab640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87ac154e70 0x7f87ac14eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:00.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.138+0000 7f87b29ab640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87ac1524d0 0x7f87ac153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:00.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.138+0000 7f87b29ab640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f87ac15afd0 con 0x7f87ac0ce070 2024-04-22T06:31:00.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.138+0000 7f87b29ab640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f87ac15bea0 con 0x7f87ac1524d0 2024-04-22T06:31:00.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.138+0000 7f87b29ab640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f87ac155250 con 0x7f87ac154e70 2024-04-22T06:31:00.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.138+0000 7f87abfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87ac0ce070 0x7f87ac1e7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:00.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.139+0000 7f87abfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87ac0ce070 0x7f87ac1e7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54778/0 (socket says 172.21.15.44:54778) 2024-04-22T06:31:00.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.139+0000 7f87abfff640 1 -- 172.21.15.44:0/1129693576 learned_addr learned my addr 172.21.15.44:0/1129693576 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:00.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.139+0000 7f87ab7fe640 1 --2- 172.21.15.44:0/1129693576 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87ac154e70 0x7f87ac14eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:00.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.139+0000 7f87b0f48640 1 --2- 172.21.15.44:0/1129693576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87ac1524d0 0x7f87ac153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:00.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.139+0000 7f87abfff640 1 -- 172.21.15.44:0/1129693576 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87ac154e70 msgr2=0x7f87ac14eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:00.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.139+0000 7f87abfff640 1 --2- 172.21.15.44:0/1129693576 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87ac154e70 0x7f87ac14eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:00.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.139+0000 7f87abfff640 1 -- 172.21.15.44:0/1129693576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87ac1524d0 msgr2=0x7f87ac153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:00.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.139+0000 7f87abfff640 1 --2- 172.21.15.44:0/1129693576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87ac1524d0 0x7f87ac153d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:00.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.139+0000 7f87abfff640 1 -- 172.21.15.44:0/1129693576 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87ac152980 con 0x7f87ac0ce070 2024-04-22T06:31:00.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.139+0000 7f87abfff640 1 --2- 172.21.15.44:0/1129693576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87ac0ce070 0x7f87ac1e7890 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8794034040 tx=0x7f87940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:00.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.139+0000 7f87a8ff9640 1 -- 172.21.15.44:0/1129693576 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8794032630 con 0x7f87ac0ce070 2024-04-22T06:31:00.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.139+0000 7f87a8ff9640 1 -- 172.21.15.44:0/1129693576 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87940327d0 con 0x7f87ac0ce070 2024-04-22T06:31:00.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.139+0000 7f87a8ff9640 1 -- 172.21.15.44:0/1129693576 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8794032bc0 con 0x7f87ac0ce070 2024-04-22T06:31:00.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.139+0000 7f87b29ab640 1 -- 172.21.15.44:0/1129693576 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87ac152c60 con 0x7f87ac0ce070 2024-04-22T06:31:00.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.139+0000 7f87b29ab640 1 -- 172.21.15.44:0/1129693576 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87ac153220 con 0x7f87ac0ce070 2024-04-22T06:31:00.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.140+0000 7f87b29ab640 1 -- 172.21.15.44:0/1129693576 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87ac1533f0 con 0x7f87ac0ce070 2024-04-22T06:31:00.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.141+0000 7f87a8ff9640 1 -- 172.21.15.44:0/1129693576 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f879404d3f0 con 0x7f87ac0ce070 2024-04-22T06:31:00.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.141+0000 7f87a8ff9640 1 --2- 172.21.15.44:0/1129693576 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f878c06c8f0 0x7f878c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:00.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.141+0000 7f87a8ff9640 1 -- 172.21.15.44:0/1129693576 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f879402f030 con 0x7f87ac0ce070 2024-04-22T06:31:00.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.141+0000 7f87ab7fe640 1 --2- 172.21.15.44:0/1129693576 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f878c06c8f0 0x7f878c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:00.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.142+0000 7f87ab7fe640 1 --2- 172.21.15.44:0/1129693576 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f878c06c8f0 0x7f878c06edb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f87ac0c9ed0 tx=0x7f879c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:00.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.146+0000 7f87a8ff9640 1 -- 172.21.15.44:0/1129693576 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8794040070 con 0x7f87ac0ce070 2024-04-22T06:31:00.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.335+0000 7f87b29ab640 1 -- 172.21.15.44:0/1129693576 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f87ac0c87c0 con 0x7f87ac0ce070 2024-04-22T06:31:00.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.336+0000 7f87a8ff9640 1 -- 172.21.15.44:0/1129693576 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12836 (secure 0 0 0) 0x7f879407b440 con 0x7f87ac0ce070 2024-04-22T06:31:00.335 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:31:00.335 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":17,"fsid":"9b0a34ca-403d-4513-8a66-d3749ba559aa","created":"2024-04-22T06:30:42.334581+0000","modified":"2024-04-22T06:30:56.957495+0000","last_up_change":"2024-04-22T06:30:52.942257+0000","last_in_change":"2024-04-22T06:30:44.467727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T06:30:54.560256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2b4a6692-53c9-472d-b2f3-1dd29a6679ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6817","nonce":2220268493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6819","nonce":2220268493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6823","nonce":2220268493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6821","nonce":2220268493}]},"public_addr":"172.21.15.44:6817/2220268493","cluster_addr":"172.21.15.44:6819/2220268493","heartbeat_back_addr":"172.21.15.44:6823/2220268493","heartbeat_front_addr":"172.21.15.44:6821/2220268493","state":["exists","up"]},{"osd":1,"uuid":"70a6370a-7c0a-4d64-a0e4-b2e726b4f291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6809","nonce":1278678977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6811","nonce":1278678977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6815","nonce":1278678977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6813","nonce":1278678977}]},"public_addr":"172.21.15.44:6809/1278678977","cluster_addr":"172.21.15.44:6811/1278678977","heartbeat_back_addr":"172.21.15.44:6815/1278678977","heartbeat_front_addr":"172.21.15.44:6813/1278678977","state":["exists","up"]},{"osd":2,"uuid":"f2e51a62-ca91-470d-bd5a-5710c61deee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6801","nonce":2544087903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6803","nonce":2544087903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6807","nonce":2544087903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6805","nonce":2544087903}]},"public_addr":"172.21.15.44:6801/2544087903","cluster_addr":"172.21.15.44:6803/2544087903","heartbeat_back_addr":"172.21.15.44:6807/2544087903","heartbeat_front_addr":"172.21.15.44:6805/2544087903","state":["exists","up"]},{"osd":3,"uuid":"bc514203-9af9-4f1f-b18e-765bdd5de6e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6825","nonce":1692928109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6827","nonce":1692928109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6831","nonce":1692928109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6829","nonce":1692928109}]},"public_addr":"172.21.15.44:6825/1692928109","cluster_addr":"172.21.15.44:6827/1692928109","heartbeat_back_addr":"172.21.15.44:6831/1692928109","heartbeat_front_addr":"172.21.15.44:6829/1692928109","state":["exists","up"]},{"osd":4,"uuid":"357c439a-6e08-4c19-ac18-b68e6c890497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6820","nonce":3802577427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6824","nonce":3802577427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6831","nonce":3802577427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6828","nonce":3802577427}]},"public_addr":"172.21.15.70:6820/3802577427","cluster_addr":"172.21.15.70:6824/3802577427","heartbeat_back_addr":"172.21.15.70:6831/3802577427","heartbeat_front_addr":"172.21.15.70:6828/3802577427","state":["exists","up"]},{"osd":5,"uuid":"952fb88f-d56e-4305-bfa8-d32674332dfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6801","nonce":664757345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6803","nonce":664757345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6807","nonce":664757345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6805","nonce":664757345}]},"public_addr":"172.21.15.70:6801/664757345","cluster_addr":"172.21.15.70:6803/664757345","heartbeat_back_addr":"172.21.15.70:6807/664757345","heartbeat_front_addr":"172.21.15.70:6805/664757345","state":["exists","up"]},{"osd":6,"uuid":"74399b3d-fac2-45ab-9172-2cd95b02450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6817","nonce":1623253482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6821","nonce":1623253482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6827","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6829","nonce":1623253482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6825","nonce":1623253482}]},"public_addr":"172.21.15.70:6817/1623253482","cluster_addr":"172.21.15.70:6821/1623253482","heartbeat_back_addr":"172.21.15.70:6829/1623253482","heartbeat_front_addr":"172.21.15.70:6825/1623253482","state":["exists","up"]},{"osd":7,"uuid":"8f35508d-6fcc-43ff-8ee7-0498a7ab3b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6809","nonce":1551180300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6811","nonce":1551180300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6815","nonce":1551180300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6813","nonce":1551180300}]},"public_addr":"172.21.15.70:6809/1551180300","cluster_addr":"172.21.15.70:6811/1551180300","heartbeat_back_addr":"172.21.15.70:6815/1551180300","heartbeat_front_addr":"172.21.15.70:6813/1551180300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.612502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.653756+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.521169+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.649955+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.640231+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.686638+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.642199+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.700360+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T06:31:00.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.338+0000 7f87b29ab640 1 -- 172.21.15.44:0/1129693576 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f878c06c8f0 msgr2=0x7f878c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:00.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.338+0000 7f87b29ab640 1 --2- 172.21.15.44:0/1129693576 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f878c06c8f0 0x7f878c06edb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f87ac0c9ed0 tx=0x7f879c009040 comp rx=0 tx=0).stop 2024-04-22T06:31:00.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.338+0000 7f87b29ab640 1 -- 172.21.15.44:0/1129693576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87ac0ce070 msgr2=0x7f87ac1e7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:00.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.338+0000 7f87b29ab640 1 --2- 172.21.15.44:0/1129693576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87ac0ce070 0x7f87ac1e7890 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8794034040 tx=0x7f87940046c0 comp rx=0 tx=0).stop 2024-04-22T06:31:00.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.338+0000 7f87b29ab640 1 -- 172.21.15.44:0/1129693576 shutdown_connections 2024-04-22T06:31:00.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.338+0000 7f87b29ab640 1 --2- 172.21.15.44:0/1129693576 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f878c06c8f0 0x7f878c06edb0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:00.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.338+0000 7f87b29ab640 1 --2- 172.21.15.44:0/1129693576 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87ac154e70 0x7f87ac14eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:00.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.338+0000 7f87b29ab640 1 --2- 172.21.15.44:0/1129693576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87ac1524d0 0x7f87ac153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:00.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.338+0000 7f87b29ab640 1 --2- 172.21.15.44:0/1129693576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87ac0ce070 0x7f87ac1e7890 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:00.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.338+0000 7f87b29ab640 1 -- 172.21.15.44:0/1129693576 >> 172.21.15.44:0/1129693576 conn(0x7f87ac067f80 msgr2=0x7f87ac0c9f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:00.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.338+0000 7f87b29ab640 1 -- 172.21.15.44:0/1129693576 shutdown_connections 2024-04-22T06:31:00.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.338+0000 7f87b29ab640 1 -- 172.21.15.44:0/1129693576 wait complete. 2024-04-22T06:31:00.347 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T06:30:54.560256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T06:31:00.348 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T06:31:00.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.452+0000 7f137f1b1640 1 Processor -- start 2024-04-22T06:31:00.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.452+0000 7f137f1b1640 1 -- start start 2024-04-22T06:31:00.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.452+0000 7f137f1b1640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1378154f00 0x7f13780cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:00.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.452+0000 7f137f1b1640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f13781552e0 0x7f137815c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:00.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.452+0000 7f137f1b1640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f137806d470 0x7f13780d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:00.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.452+0000 7f137f1b1640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f13780c6740 con 0x7f1378154f00 2024-04-22T06:31:00.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.452+0000 7f137f1b1640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f13780c5b30 con 0x7f13781552e0 2024-04-22T06:31:00.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.452+0000 7f137f1b1640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f13780c5e10 con 0x7f137806d470 2024-04-22T06:31:00.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.453+0000 7f137cf4d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1378154f00 0x7f13780cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:00.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.453+0000 7f137cf4d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1378154f00 0x7f13780cb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54790/0 (socket says 172.21.15.44:54790) 2024-04-22T06:31:00.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.453+0000 7f137cf4d640 1 -- 172.21.15.44:0/2958838705 learned_addr learned my addr 172.21.15.44:0/2958838705 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:00.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.453+0000 7f137d74e640 1 --2- 172.21.15.44:0/2958838705 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f137806d470 0x7f13780d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:00.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.453+0000 7f1377fff640 1 --2- 172.21.15.44:0/2958838705 >> v2:172.21.15.70:3300/0 conn(0x7f13781552e0 0x7f137815c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:00.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.453+0000 7f137cf4d640 1 -- 172.21.15.44:0/2958838705 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f137806d470 msgr2=0x7f13780d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:00.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.453+0000 7f137cf4d640 1 --2- 172.21.15.44:0/2958838705 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f137806d470 0x7f13780d0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:00.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.453+0000 7f137cf4d640 1 -- 172.21.15.44:0/2958838705 >> v2:172.21.15.70:3300/0 conn(0x7f13781552e0 msgr2=0x7f137815c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:00.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.453+0000 7f137cf4d640 1 --2- 172.21.15.44:0/2958838705 >> v2:172.21.15.70:3300/0 conn(0x7f13781552e0 0x7f137815c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:00.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.453+0000 7f137cf4d640 1 -- 172.21.15.44:0/2958838705 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13780c60f0 con 0x7f1378154f00 2024-04-22T06:31:00.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.454+0000 7f137cf4d640 1 --2- 172.21.15.44:0/2958838705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1378154f00 0x7f13780cb430 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f13780ce560 tx=0x7f136c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a992e246ae423702 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:00.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.454+0000 7f13777fe640 1 -- 172.21.15.44:0/2958838705 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136c035020 con 0x7f1378154f00 2024-04-22T06:31:00.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.454+0000 7f13777fe640 1 -- 172.21.15.44:0/2958838705 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f136c031870 con 0x7f1378154f00 2024-04-22T06:31:00.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.454+0000 7f13777fe640 1 -- 172.21.15.44:0/2958838705 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136c031c40 con 0x7f1378154f00 2024-04-22T06:31:00.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.454+0000 7f137f1b1640 1 -- 172.21.15.44:0/2958838705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1378154f00 msgr2=0x7f13780cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:00.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.454+0000 7f137f1b1640 1 --2- 172.21.15.44:0/2958838705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1378154f00 0x7f13780cb430 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f13780ce560 tx=0x7f136c02b910 comp rx=0 tx=0).stop 2024-04-22T06:31:00.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.454+0000 7f137f1b1640 1 -- 172.21.15.44:0/2958838705 shutdown_connections 2024-04-22T06:31:00.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.454+0000 7f137f1b1640 1 --2- 172.21.15.44:0/2958838705 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f137806d470 0x7f13780d0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:00.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.454+0000 7f137f1b1640 1 --2- 172.21.15.44:0/2958838705 >> v2:172.21.15.70:3300/0 conn(0x7f13781552e0 0x7f137815c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:00.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.454+0000 7f137f1b1640 1 --2- 172.21.15.44:0/2958838705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1378154f00 0x7f13780cb430 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:00.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.454+0000 7f137f1b1640 1 -- 172.21.15.44:0/2958838705 >> 172.21.15.44:0/2958838705 conn(0x7f13780733c0 msgr2=0x7f13780737d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:00.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.454+0000 7f137f1b1640 1 -- 172.21.15.44:0/2958838705 shutdown_connections 2024-04-22T06:31:00.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.454+0000 7f137f1b1640 1 -- 172.21.15.44:0/2958838705 wait complete. 2024-04-22T06:31:00.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.455+0000 7f137f1b1640 1 Processor -- start 2024-04-22T06:31:00.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.455+0000 7f137f1b1640 1 -- start start 2024-04-22T06:31:00.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.456+0000 7f137f1b1640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f137806d470 0x7f13780b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:00.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.456+0000 7f137f1b1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1378154f00 0x7f13780cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:00.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.456+0000 7f137f1b1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13781552e0 0x7f1378159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:00.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.456+0000 7f137f1b1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f13780c6d50 con 0x7f1378154f00 2024-04-22T06:31:00.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.456+0000 7f137f1b1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f13780c6bd0 con 0x7f13781552e0 2024-04-22T06:31:00.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.456+0000 7f137f1b1640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f137815cb20 con 0x7f137806d470 2024-04-22T06:31:00.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.456+0000 7f1377fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1378154f00 0x7f13780cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:00.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.456+0000 7f1377fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1378154f00 0x7f13780cdb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54802/0 (socket says 172.21.15.44:54802) 2024-04-22T06:31:00.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.456+0000 7f1377fff640 1 -- 172.21.15.44:0/3609018157 learned_addr learned my addr 172.21.15.44:0/3609018157 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:00.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.456+0000 7f137cf4d640 1 --2- 172.21.15.44:0/3609018157 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f137806d470 0x7f13780b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:00.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.456+0000 7f137d74e640 1 --2- 172.21.15.44:0/3609018157 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13781552e0 0x7f1378159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:00.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.457+0000 7f1377fff640 1 -- 172.21.15.44:0/3609018157 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f137806d470 msgr2=0x7f13780b5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:00.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.457+0000 7f1377fff640 1 --2- 172.21.15.44:0/3609018157 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f137806d470 0x7f13780b5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:00.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.457+0000 7f1377fff640 1 -- 172.21.15.44:0/3609018157 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13781552e0 msgr2=0x7f1378159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:00.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.457+0000 7f1377fff640 1 --2- 172.21.15.44:0/3609018157 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13781552e0 0x7f1378159690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:00.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.457+0000 7f1377fff640 1 -- 172.21.15.44:0/3609018157 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13780cd6e0 con 0x7f1378154f00 2024-04-22T06:31:00.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.457+0000 7f1377fff640 1 --2- 172.21.15.44:0/3609018157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1378154f00 0x7f13780cdb90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f136400c7f0 tx=0x7f136400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:00.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.457+0000 7f13757fa640 1 -- 172.21.15.44:0/3609018157 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1364015070 con 0x7f1378154f00 2024-04-22T06:31:00.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.457+0000 7f13757fa640 1 -- 172.21.15.44:0/3609018157 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1364004220 con 0x7f1378154f00 2024-04-22T06:31:00.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.457+0000 7f137f1b1640 1 -- 172.21.15.44:0/3609018157 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13780b49a0 con 0x7f1378154f00 2024-04-22T06:31:00.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.457+0000 7f13757fa640 1 -- 172.21.15.44:0/3609018157 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1364003610 con 0x7f1378154f00 2024-04-22T06:31:00.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.457+0000 7f137f1b1640 1 -- 172.21.15.44:0/3609018157 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13780b5680 con 0x7f1378154f00 2024-04-22T06:31:00.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.458+0000 7f137f1b1640 1 -- 172.21.15.44:0/3609018157 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1378068cc0 con 0x7f1378154f00 2024-04-22T06:31:00.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.459+0000 7f13757fa640 1 -- 172.21.15.44:0/3609018157 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f13640039c0 con 0x7f1378154f00 2024-04-22T06:31:00.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.459+0000 7f13757fa640 1 --2- 172.21.15.44:0/3609018157 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f135c06c8f0 0x7f135c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:00.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.459+0000 7f13757fa640 1 -- 172.21.15.44:0/3609018157 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f136408ba30 con 0x7f1378154f00 2024-04-22T06:31:00.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.459+0000 7f137cf4d640 1 --2- 172.21.15.44:0/3609018157 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f135c06c8f0 0x7f135c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:00.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.460+0000 7f137cf4d640 1 --2- 172.21.15.44:0/3609018157 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f135c06c8f0 0x7f135c06edb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f136c036000 tx=0x7f136c03c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:00.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.464+0000 7f13757fa640 1 -- 172.21.15.44:0/3609018157 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1364059b60 con 0x7f1378154f00 2024-04-22T06:31:00.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.647+0000 7f137f1b1640 1 -- 172.21.15.44:0/3609018157 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f13781517d0 con 0x7f1378154f00 2024-04-22T06:31:00.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.648+0000 7f13757fa640 1 -- 172.21.15.44:0/3609018157 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) ==== 93+0+10 (secure 0 0 0) 0x7f136405de60 con 0x7f1378154f00 2024-04-22T06:31:00.647 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-04-22T06:31:00.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.650+0000 7f137f1b1640 1 -- 172.21.15.44:0/3609018157 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f135c06c8f0 msgr2=0x7f135c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:00.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.650+0000 7f137f1b1640 1 --2- 172.21.15.44:0/3609018157 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f135c06c8f0 0x7f135c06edb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f136c036000 tx=0x7f136c03c040 comp rx=0 tx=0).stop 2024-04-22T06:31:00.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.650+0000 7f137f1b1640 1 -- 172.21.15.44:0/3609018157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1378154f00 msgr2=0x7f13780cdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:00.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.650+0000 7f137f1b1640 1 --2- 172.21.15.44:0/3609018157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1378154f00 0x7f13780cdb90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f136400c7f0 tx=0x7f136400ccb0 comp rx=0 tx=0).stop 2024-04-22T06:31:00.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.650+0000 7f137f1b1640 1 -- 172.21.15.44:0/3609018157 shutdown_connections 2024-04-22T06:31:00.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.650+0000 7f137f1b1640 1 --2- 172.21.15.44:0/3609018157 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f135c06c8f0 0x7f135c06edb0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:00.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.650+0000 7f137f1b1640 1 --2- 172.21.15.44:0/3609018157 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13781552e0 0x7f1378159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:00.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.650+0000 7f137f1b1640 1 --2- 172.21.15.44:0/3609018157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1378154f00 0x7f13780cdb90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:00.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.650+0000 7f137f1b1640 1 --2- 172.21.15.44:0/3609018157 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f137806d470 0x7f13780b5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:00.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.650+0000 7f137f1b1640 1 -- 172.21.15.44:0/3609018157 >> 172.21.15.44:0/3609018157 conn(0x7f13780733c0 msgr2=0x7f13781e4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:00.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.650+0000 7f137f1b1640 1 -- 172.21.15.44:0/3609018157 shutdown_connections 2024-04-22T06:31:00.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.650+0000 7f137f1b1640 1 -- 172.21.15.44:0/3609018157 wait complete. 2024-04-22T06:31:00.659 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T06:31:00.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.760+0000 7ff90bff4640 1 Processor -- start 2024-04-22T06:31:00.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.761+0000 7ff90bff4640 1 -- start start 2024-04-22T06:31:00.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.761+0000 7ff90bff4640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ff904154f40 0x7ff904073a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:00.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.761+0000 7ff90bff4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff904155320 0x7ff9040ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:00.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.761+0000 7ff90bff4640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff90406c560 con 0x7ff904074260 2024-04-22T06:31:00.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.761+0000 7ff90bff4640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7ff90415bfa0 con 0x7ff904154f40 2024-04-22T06:31:00.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.761+0000 7ff90bff4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7ff90415b420 con 0x7ff904155320 2024-04-22T06:31:00.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.761+0000 7ff909d90640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ff904154f40 0x7ff904073a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:00.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.761+0000 7ff909d90640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ff904154f40 0x7ff904073a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54818/0 (socket says 172.21.15.44:54818) 2024-04-22T06:31:00.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.761+0000 7ff90a591640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7ff904074260 0x7ff90415ec00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:47808/0 (socket says 172.21.15.44:47808) 2024-04-22T06:31:00.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.761+0000 7ff909d90640 1 -- 172.21.15.44:0/261399517 learned_addr learned my addr 172.21.15.44:0/261399517 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:00.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.762+0000 7ff90958f640 1 --2- 172.21.15.44:0/261399517 >> v2:172.21.15.70:3300/0 conn(0x7ff904155320 0x7ff9040ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:00.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.762+0000 7ff909d90640 1 -- 172.21.15.44:0/261399517 >> v2:172.21.15.70:3300/0 conn(0x7ff904155320 msgr2=0x7ff9040ce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:00.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.762+0000 7ff909d90640 1 --2- 172.21.15.44:0/261399517 >> v2:172.21.15.70:3300/0 conn(0x7ff904155320 0x7ff9040ce6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:00.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.762+0000 7ff909d90640 1 -- 172.21.15.44:0/261399517 >> v1:172.21.15.44:6789/0 conn(0x7ff904074260 legacy=0x7ff90415ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:00.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.762+0000 7ff909d90640 1 -- 172.21.15.44:0/261399517 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff90415b700 con 0x7ff904154f40 2024-04-22T06:31:00.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.762+0000 7ff90958f640 1 --2- 172.21.15.44:0/261399517 >> v2:172.21.15.70:3300/0 conn(0x7ff904155320 0x7ff9040ce6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:31:00.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.762+0000 7ff909d90640 1 --2- 172.21.15.44:0/261399517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff904154f40 0x7ff904073a10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff904071be0 tx=0x7ff90002ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f3471b43b5a5877 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:00.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.763+0000 7ff908d8e640 1 -- 172.21.15.44:0/261399517 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90002f850 con 0x7ff904154f40 2024-04-22T06:31:00.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.763+0000 7ff908d8e640 1 -- 172.21.15.44:0/261399517 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff90002f9f0 con 0x7ff904154f40 2024-04-22T06:31:00.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.763+0000 7ff908d8e640 1 -- 172.21.15.44:0/261399517 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90002fdc0 con 0x7ff904154f40 2024-04-22T06:31:00.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.763+0000 7ff90bff4640 1 -- 172.21.15.44:0/261399517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff904154f40 msgr2=0x7ff904073a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:00.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.763+0000 7ff90bff4640 1 --2- 172.21.15.44:0/261399517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff904154f40 0x7ff904073a10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff904071be0 tx=0x7ff90002ba30 comp rx=0 tx=0).stop 2024-04-22T06:31:00.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.763+0000 7ff90bff4640 1 -- 172.21.15.44:0/261399517 shutdown_connections 2024-04-22T06:31:00.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.763+0000 7ff90bff4640 1 --2- 172.21.15.44:0/261399517 >> v2:172.21.15.70:3300/0 conn(0x7ff904155320 0x7ff9040ce6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:00.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.763+0000 7ff90bff4640 1 --2- 172.21.15.44:0/261399517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff904154f40 0x7ff904073a10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:00.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.763+0000 7ff90bff4640 1 -- 172.21.15.44:0/261399517 >> 172.21.15.44:0/261399517 conn(0x7ff904074cb0 msgr2=0x7ff9040750c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:00.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.763+0000 7ff90bff4640 1 -- 172.21.15.44:0/261399517 shutdown_connections 2024-04-22T06:31:00.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.763+0000 7ff90bff4640 1 -- 172.21.15.44:0/261399517 wait complete. 2024-04-22T06:31:00.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.764+0000 7ff90bff4640 1 Processor -- start 2024-04-22T06:31:00.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.765+0000 7ff90bff4640 1 -- start start 2024-04-22T06:31:00.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.765+0000 7ff90bff4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff904074260 0x7ff9040cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:00.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.765+0000 7ff90bff4640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff904154f40 0x7ff9040ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:00.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.765+0000 7ff90bff4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9041596d0 0x7ff9041579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:00.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.765+0000 7ff90bff4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff90415e6e0 con 0x7ff9041596d0 2024-04-22T06:31:00.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.765+0000 7ff90bff4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff90415bcc0 con 0x7ff904074260 2024-04-22T06:31:00.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.765+0000 7ff90bff4640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ff904073f50 con 0x7ff904154f40 2024-04-22T06:31:00.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.765+0000 7ff90a591640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9041596d0 0x7ff9041579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:00.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.765+0000 7ff90a591640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9041596d0 0x7ff9041579e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54834/0 (socket says 172.21.15.44:54834) 2024-04-22T06:31:00.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.765+0000 7ff90a591640 1 -- 172.21.15.44:0/2588347001 learned_addr learned my addr 172.21.15.44:0/2588347001 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:00.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.765+0000 7ff90958f640 1 --2- 172.21.15.44:0/2588347001 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff904154f40 0x7ff9040ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:00.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.766+0000 7ff909d90640 1 --2- 172.21.15.44:0/2588347001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff904074260 0x7ff9040cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:00.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.766+0000 7ff90a591640 1 -- 172.21.15.44:0/2588347001 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff904154f40 msgr2=0x7ff9040ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:00.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.766+0000 7ff90a591640 1 --2- 172.21.15.44:0/2588347001 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff904154f40 0x7ff9040ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:00.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.766+0000 7ff90a591640 1 -- 172.21.15.44:0/2588347001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff904074260 msgr2=0x7ff9040cb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:00.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.766+0000 7ff90a591640 1 --2- 172.21.15.44:0/2588347001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff904074260 0x7ff9040cb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:00.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.766+0000 7ff90a591640 1 -- 172.21.15.44:0/2588347001 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9040cbb50 con 0x7ff9041596d0 2024-04-22T06:31:00.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.766+0000 7ff909d90640 1 --2- 172.21.15.44:0/2588347001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff904074260 0x7ff9040cb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:31:00.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.766+0000 7ff90a591640 1 --2- 172.21.15.44:0/2588347001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9041596d0 0x7ff9041579e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc00c780 tx=0x7ff8fc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:00.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.766+0000 7ff8fa7fc640 1 -- 172.21.15.44:0/2588347001 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc017070 con 0x7ff9041596d0 2024-04-22T06:31:00.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.766+0000 7ff8fa7fc640 1 -- 172.21.15.44:0/2588347001 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8fc00a640 con 0x7ff9041596d0 2024-04-22T06:31:00.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.766+0000 7ff90bff4640 1 -- 172.21.15.44:0/2588347001 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9040ca820 con 0x7ff9041596d0 2024-04-22T06:31:00.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.766+0000 7ff8fa7fc640 1 -- 172.21.15.44:0/2588347001 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc00aa30 con 0x7ff9041596d0 2024-04-22T06:31:00.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.766+0000 7ff90bff4640 1 -- 172.21.15.44:0/2588347001 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff904156c20 con 0x7ff9041596d0 2024-04-22T06:31:00.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.767+0000 7ff90bff4640 1 -- 172.21.15.44:0/2588347001 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff904069310 con 0x7ff9041596d0 2024-04-22T06:31:00.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.768+0000 7ff8fa7fc640 1 -- 172.21.15.44:0/2588347001 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff8fc01a070 con 0x7ff9041596d0 2024-04-22T06:31:00.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.768+0000 7ff8fa7fc640 1 --2- 172.21.15.44:0/2588347001 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7ff8e806c920 0x7ff8e806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:00.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.768+0000 7ff8fa7fc640 1 -- 172.21.15.44:0/2588347001 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7ff8fc08acf0 con 0x7ff9041596d0 2024-04-22T06:31:00.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.768+0000 7ff909d90640 1 --2- 172.21.15.44:0/2588347001 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7ff8e806c920 0x7ff8e806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:00.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.769+0000 7ff909d90640 1 --2- 172.21.15.44:0/2588347001 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7ff8e806c920 0x7ff8e806ede0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff900036000 tx=0x7ff900008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:00.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.772+0000 7ff8fa7fc640 1 -- 172.21.15.44:0/2588347001 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff8fc058e20 con 0x7ff9041596d0 2024-04-22T06:31:00.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.995+0000 7ff90bff4640 1 -- 172.21.15.44:0/2588347001 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9040c96b0 con 0x7ff9041596d0 2024-04-22T06:31:00.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.995+0000 7ff8fa7fc640 1 -- 172.21.15.44:0/2588347001 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1392 (secure 0 0 0) 0x7ff8fc05d120 con 0x7ff9041596d0 2024-04-22T06:31:00.994 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:31:00.994 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1497302315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1497302315},{"type":"v1","addr":"172.21.15.44:6834","nonce":1497302315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-22T06:31:00.994 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 2 2024-04-22T06:31:00.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.997+0000 7ff90bff4640 1 -- 172.21.15.44:0/2588347001 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7ff8e806c920 msgr2=0x7ff8e806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:00.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.997+0000 7ff90bff4640 1 --2- 172.21.15.44:0/2588347001 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7ff8e806c920 0x7ff8e806ede0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff900036000 tx=0x7ff900008030 comp rx=0 tx=0).stop 2024-04-22T06:31:00.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.997+0000 7ff90bff4640 1 -- 172.21.15.44:0/2588347001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9041596d0 msgr2=0x7ff9041579e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:00.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.997+0000 7ff90bff4640 1 --2- 172.21.15.44:0/2588347001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9041596d0 0x7ff9041579e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc00c780 tx=0x7ff8fc00cc40 comp rx=0 tx=0).stop 2024-04-22T06:31:00.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.997+0000 7ff90bff4640 1 -- 172.21.15.44:0/2588347001 shutdown_connections 2024-04-22T06:31:00.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.997+0000 7ff90bff4640 1 --2- 172.21.15.44:0/2588347001 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7ff8e806c920 0x7ff8e806ede0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:00.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.997+0000 7ff90bff4640 1 --2- 172.21.15.44:0/2588347001 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff904154f40 0x7ff9040ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:00.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.997+0000 7ff90bff4640 1 --2- 172.21.15.44:0/2588347001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9041596d0 0x7ff9041579e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:00.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.997+0000 7ff90bff4640 1 --2- 172.21.15.44:0/2588347001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff904074260 0x7ff9040cb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:00.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.997+0000 7ff90bff4640 1 -- 172.21.15.44:0/2588347001 >> 172.21.15.44:0/2588347001 conn(0x7ff904074cb0 msgr2=0x7ff904153f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:00.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.997+0000 7ff90bff4640 1 -- 172.21.15.44:0/2588347001 shutdown_connections 2024-04-22T06:31:00.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:00.997+0000 7ff90bff4640 1 -- 172.21.15.44:0/2588347001 wait complete. 2024-04-22T06:31:03.007 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T06:31:03.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.122+0000 7fd777225640 1 Processor -- start 2024-04-22T06:31:03.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.122+0000 7fd777225640 1 -- start start 2024-04-22T06:31:03.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.123+0000 7fd777225640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd770155240 0x7fd77015f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:03.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.123+0000 7fd777225640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd7700cb320 0x7fd7700d0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:03.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.123+0000 7fd777225640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd7700d1400 0x7fd7700cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:03.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.123+0000 7fd777225640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fd77015f610 con 0x7fd7700d1400 2024-04-22T06:31:03.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.123+0000 7fd777225640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fd77015c060 con 0x7fd7700cb320 2024-04-22T06:31:03.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.123+0000 7fd777225640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd77015b200 con 0x7fd770155240 2024-04-22T06:31:03.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.123+0000 7fd7757c2640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd7700d1400 0x7fd7700cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:03.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.123+0000 7fd7757c2640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd7700d1400 0x7fd7700cc480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60514/0 (socket says 172.21.15.44:60514) 2024-04-22T06:31:03.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.123+0000 7fd7757c2640 1 -- 172.21.15.44:0/2954559499 learned_addr learned my addr 172.21.15.44:0/2954559499 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:03.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.123+0000 7fd7757c2640 1 -- 172.21.15.44:0/2954559499 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd770155240 msgr2=0x7fd77015f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:03.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.123+0000 7fd774fc1640 1 --2- 172.21.15.44:0/2954559499 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd770155240 0x7fd77015f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:03.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.123+0000 7fd76ffff640 1 --2- 172.21.15.44:0/2954559499 >> v2:172.21.15.70:3300/0 conn(0x7fd7700cb320 0x7fd7700d0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:03.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.123+0000 7fd7757c2640 1 --2- 172.21.15.44:0/2954559499 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd770155240 0x7fd77015f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:03.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.123+0000 7fd7757c2640 1 -- 172.21.15.44:0/2954559499 >> v2:172.21.15.70:3300/0 conn(0x7fd7700cb320 msgr2=0x7fd7700d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:03.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.123+0000 7fd7757c2640 1 --2- 172.21.15.44:0/2954559499 >> v2:172.21.15.70:3300/0 conn(0x7fd7700cb320 0x7fd7700d0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:03.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.123+0000 7fd7757c2640 1 -- 172.21.15.44:0/2954559499 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd77015b4e0 con 0x7fd7700d1400 2024-04-22T06:31:03.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.123+0000 7fd774fc1640 1 --2- 172.21.15.44:0/2954559499 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd770155240 0x7fd77015f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:31:03.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.123+0000 7fd76ffff640 1 --2- 172.21.15.44:0/2954559499 >> v2:172.21.15.70:3300/0 conn(0x7fd7700cb320 0x7fd7700d0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:31:03.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.124+0000 7fd7757c2640 1 --2- 172.21.15.44:0/2954559499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7700d1400 0x7fd7700cc480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd77015b410 tx=0x7fd76802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97b4e2cd6a8bd85 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:03.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.124+0000 7fd76f7fe640 1 -- 172.21.15.44:0/2954559499 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76802ca50 con 0x7fd7700d1400 2024-04-22T06:31:03.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.124+0000 7fd76f7fe640 1 -- 172.21.15.44:0/2954559499 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd76802cbf0 con 0x7fd7700d1400 2024-04-22T06:31:03.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.124+0000 7fd76f7fe640 1 -- 172.21.15.44:0/2954559499 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7680343f0 con 0x7fd7700d1400 2024-04-22T06:31:03.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.125+0000 7fd777225640 1 -- 172.21.15.44:0/2954559499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7700d1400 msgr2=0x7fd7700cc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:03.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.125+0000 7fd777225640 1 --2- 172.21.15.44:0/2954559499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7700d1400 0x7fd7700cc480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd77015b410 tx=0x7fd76802beb0 comp rx=0 tx=0).stop 2024-04-22T06:31:03.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.125+0000 7fd777225640 1 -- 172.21.15.44:0/2954559499 shutdown_connections 2024-04-22T06:31:03.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.125+0000 7fd777225640 1 --2- 172.21.15.44:0/2954559499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7700d1400 0x7fd7700cc480 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:03.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.125+0000 7fd777225640 1 --2- 172.21.15.44:0/2954559499 >> v2:172.21.15.70:3300/0 conn(0x7fd7700cb320 0x7fd7700d0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:03.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.125+0000 7fd777225640 1 --2- 172.21.15.44:0/2954559499 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd770155240 0x7fd77015f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:03.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.125+0000 7fd777225640 1 -- 172.21.15.44:0/2954559499 >> 172.21.15.44:0/2954559499 conn(0x7fd7700750d0 msgr2=0x7fd7700754e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:03.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.125+0000 7fd777225640 1 -- 172.21.15.44:0/2954559499 shutdown_connections 2024-04-22T06:31:03.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.125+0000 7fd777225640 1 -- 172.21.15.44:0/2954559499 wait complete. 2024-04-22T06:31:03.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.126+0000 7fd777225640 1 Processor -- start 2024-04-22T06:31:03.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.126+0000 7fd777225640 1 -- start start 2024-04-22T06:31:03.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.127+0000 7fd777225640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7700cb320 0x7fd7700b5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:03.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.127+0000 7fd777225640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7700d1400 0x7fd7701483e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:03.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.127+0000 7fd777225640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd770155240 0x7fd770148920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:03.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.127+0000 7fd777225640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd77015bd80 con 0x7fd7700d1400 2024-04-22T06:31:03.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.127+0000 7fd777225640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd77015baa0 con 0x7fd7700cb320 2024-04-22T06:31:03.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.127+0000 7fd777225640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd770155620 con 0x7fd770155240 2024-04-22T06:31:03.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.127+0000 7fd76ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7700d1400 0x7fd7701483e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:03.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.127+0000 7fd76ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7700d1400 0x7fd7701483e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60516/0 (socket says 172.21.15.44:60516) 2024-04-22T06:31:03.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.127+0000 7fd76ffff640 1 -- 172.21.15.44:0/1926408159 learned_addr learned my addr 172.21.15.44:0/1926408159 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:03.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.127+0000 7fd7757c2640 1 --2- 172.21.15.44:0/1926408159 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd770155240 0x7fd770148920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:03.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.127+0000 7fd774fc1640 1 --2- 172.21.15.44:0/1926408159 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7700cb320 0x7fd7700b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:03.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.127+0000 7fd76ffff640 1 -- 172.21.15.44:0/1926408159 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd770155240 msgr2=0x7fd770148920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:03.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.127+0000 7fd76ffff640 1 --2- 172.21.15.44:0/1926408159 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd770155240 0x7fd770148920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:03.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.127+0000 7fd76ffff640 1 -- 172.21.15.44:0/1926408159 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7700cb320 msgr2=0x7fd7700b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:03.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.127+0000 7fd76ffff640 1 --2- 172.21.15.44:0/1926408159 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7700cb320 0x7fd7700b5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:03.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.127+0000 7fd76ffff640 1 -- 172.21.15.44:0/1926408159 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7700cab90 con 0x7fd7700d1400 2024-04-22T06:31:03.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.127+0000 7fd7757c2640 1 --2- 172.21.15.44:0/1926408159 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd770155240 0x7fd770148920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:31:03.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.127+0000 7fd774fc1640 1 --2- 172.21.15.44:0/1926408159 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7700cb320 0x7fd7700b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:31:03.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.128+0000 7fd76ffff640 1 --2- 172.21.15.44:0/1926408159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7700d1400 0x7fd7701483e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd75c00c7f0 tx=0x7fd75c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:03.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.128+0000 7fd76d7fa640 1 -- 172.21.15.44:0/1926408159 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75c015070 con 0x7fd7700d1400 2024-04-22T06:31:03.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.128+0000 7fd76d7fa640 1 -- 172.21.15.44:0/1926408159 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd75c007a30 con 0x7fd7700d1400 2024-04-22T06:31:03.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.128+0000 7fd777225640 1 -- 172.21.15.44:0/1926408159 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd77015c280 con 0x7fd7700d1400 2024-04-22T06:31:03.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.128+0000 7fd76d7fa640 1 -- 172.21.15.44:0/1926408159 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75c007e00 con 0x7fd7700d1400 2024-04-22T06:31:03.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.128+0000 7fd777225640 1 -- 172.21.15.44:0/1926408159 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd77014ab30 con 0x7fd7700d1400 2024-04-22T06:31:03.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.129+0000 7fd777225640 1 -- 172.21.15.44:0/1926408159 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7700c6750 con 0x7fd7700d1400 2024-04-22T06:31:03.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.129+0000 7fd76d7fa640 1 -- 172.21.15.44:0/1926408159 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd75c018070 con 0x7fd7700d1400 2024-04-22T06:31:03.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.130+0000 7fd76d7fa640 1 --2- 172.21.15.44:0/1926408159 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd75406c920 0x7fd75406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:03.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.130+0000 7fd76d7fa640 1 -- 172.21.15.44:0/1926408159 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fd75c08b740 con 0x7fd7700d1400 2024-04-22T06:31:03.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.130+0000 7fd774fc1640 1 --2- 172.21.15.44:0/1926408159 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd75406c920 0x7fd75406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:03.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.131+0000 7fd774fc1640 1 --2- 172.21.15.44:0/1926408159 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd75406c920 0x7fd75406ede0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd76400a990 tx=0x7fd764008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:03.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.134+0000 7fd76d7fa640 1 -- 172.21.15.44:0/1926408159 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd7700c6750 con 0x7fd7700d1400 2024-04-22T06:31:03.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.357+0000 7fd777225640 1 -- 172.21.15.44:0/1926408159 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7700cadd0 con 0x7fd7700d1400 2024-04-22T06:31:03.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.357+0000 7fd76d7fa640 1 -- 172.21.15.44:0/1926408159 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+5506 (secure 0 0 0) 0x7fd75c059ec0 con 0x7fd7700d1400 2024-04-22T06:31:03.356 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:31:03.356 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/1779271860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":1779271860},{"type":"v1","addr":"172.21.15.70:6836","nonce":1779271860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1497302315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1497302315},{"type":"v1","addr":"172.21.15.44:6834","nonce":1497302315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/2383621014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2383621014},{"type":"v1","addr":"172.21.15.44:6835","nonce":2383621014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4293,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/915960587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":915960587},{"type":"v1","addr":"172.21.15.44:6837","nonce":915960587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4305,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6838/924955785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":924955785},{"type":"v1","addr":"172.21.15.70:6838","nonce":924955785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4308,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6839/2915150040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6837","nonce":2915150040},{"type":"v1","addr":"172.21.15.70:6839","nonce":2915150040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-22T06:31:03.357 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 3 2024-04-22T06:31:03.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.359+0000 7fd777225640 1 -- 172.21.15.44:0/1926408159 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd75406c920 msgr2=0x7fd75406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:03.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.359+0000 7fd777225640 1 --2- 172.21.15.44:0/1926408159 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd75406c920 0x7fd75406ede0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd76400a990 tx=0x7fd764008040 comp rx=0 tx=0).stop 2024-04-22T06:31:03.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.359+0000 7fd777225640 1 -- 172.21.15.44:0/1926408159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7700d1400 msgr2=0x7fd7701483e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:03.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.359+0000 7fd777225640 1 --2- 172.21.15.44:0/1926408159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7700d1400 0x7fd7701483e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd75c00c7f0 tx=0x7fd75c00ccb0 comp rx=0 tx=0).stop 2024-04-22T06:31:03.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.359+0000 7fd777225640 1 -- 172.21.15.44:0/1926408159 shutdown_connections 2024-04-22T06:31:03.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.359+0000 7fd777225640 1 --2- 172.21.15.44:0/1926408159 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd75406c920 0x7fd75406ede0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:03.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.359+0000 7fd777225640 1 --2- 172.21.15.44:0/1926408159 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd770155240 0x7fd770148920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:03.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.359+0000 7fd777225640 1 --2- 172.21.15.44:0/1926408159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7700d1400 0x7fd7701483e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:03.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.359+0000 7fd777225640 1 --2- 172.21.15.44:0/1926408159 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7700cb320 0x7fd7700b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:03.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.359+0000 7fd777225640 1 -- 172.21.15.44:0/1926408159 >> 172.21.15.44:0/1926408159 conn(0x7fd7700750d0 msgr2=0x7fd77015eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:03.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.359+0000 7fd777225640 1 -- 172.21.15.44:0/1926408159 shutdown_connections 2024-04-22T06:31:03.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.359+0000 7fd777225640 1 -- 172.21.15.44:0/1926408159 wait complete. 2024-04-22T06:31:03.369 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T06:31:03.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.470+0000 7f295fb52640 1 Processor -- start 2024-04-22T06:31:03.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.470+0000 7f295fb52640 1 -- start start 2024-04-22T06:31:03.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.471+0000 7f295fb52640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2958155320 0x7f29580ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:03.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.471+0000 7f295fb52640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2958074260 0x7f29580cec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:03.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.471+0000 7f295fb52640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f295806c560 con 0x7f2958154f40 2024-04-22T06:31:03.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.471+0000 7f295fb52640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f295815bfa0 con 0x7f2958074260 2024-04-22T06:31:03.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.471+0000 7f295fb52640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f295815b420 con 0x7f2958155320 2024-04-22T06:31:03.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.471+0000 7f295e0ef640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2958074260 0x7f29580cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:03.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.471+0000 7f295e0ef640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2958074260 0x7f29580cec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60524/0 (socket says 172.21.15.44:60524) 2024-04-22T06:31:03.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.471+0000 7f295e0ef640 1 -- 172.21.15.44:0/3366897882 learned_addr learned my addr 172.21.15.44:0/3366897882 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:03.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.472+0000 7f295d0ed640 1 --2- 172.21.15.44:0/3366897882 >> v2:172.21.15.70:3300/0 conn(0x7f2958155320 0x7f29580ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:03.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.472+0000 7f295e0ef640 1 -- 172.21.15.44:0/3366897882 >> v2:172.21.15.70:3300/0 conn(0x7f2958155320 msgr2=0x7f29580ce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:03.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.472+0000 7f295e0ef640 1 --2- 172.21.15.44:0/3366897882 >> v2:172.21.15.70:3300/0 conn(0x7f2958155320 0x7f29580ce6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:03.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.472+0000 7f295e0ef640 1 -- 172.21.15.44:0/3366897882 >> v1:172.21.15.70:6789/0 conn(0x7f2958154f40 legacy=0x7f2958073a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:03.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.472+0000 7f295e0ef640 1 -- 172.21.15.44:0/3366897882 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f295815b700 con 0x7f2958074260 2024-04-22T06:31:03.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.472+0000 7f295e0ef640 1 --2- 172.21.15.44:0/3366897882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2958074260 0x7f29580cec30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2958071be0 tx=0x7f294802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da5c71916cd9b908 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:03.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.473+0000 7f295c8ec640 1 -- 172.21.15.44:0/3366897882 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294802cb20 con 0x7f2958074260 2024-04-22T06:31:03.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.473+0000 7f295c8ec640 1 -- 172.21.15.44:0/3366897882 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f294802ccc0 con 0x7f2958074260 2024-04-22T06:31:03.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.473+0000 7f295c8ec640 1 -- 172.21.15.44:0/3366897882 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2948034610 con 0x7f2958074260 2024-04-22T06:31:03.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.473+0000 7f295fb52640 1 -- 172.21.15.44:0/3366897882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2958074260 msgr2=0x7f29580cec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:03.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.473+0000 7f295fb52640 1 --2- 172.21.15.44:0/3366897882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2958074260 0x7f29580cec30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2958071be0 tx=0x7f294802bf80 comp rx=0 tx=0).stop 2024-04-22T06:31:03.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.473+0000 7f295fb52640 1 -- 172.21.15.44:0/3366897882 shutdown_connections 2024-04-22T06:31:03.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.473+0000 7f295fb52640 1 --2- 172.21.15.44:0/3366897882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2958074260 0x7f29580cec30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:03.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.473+0000 7f295fb52640 1 --2- 172.21.15.44:0/3366897882 >> v2:172.21.15.70:3300/0 conn(0x7f2958155320 0x7f29580ce6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:03.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.473+0000 7f295fb52640 1 -- 172.21.15.44:0/3366897882 >> 172.21.15.44:0/3366897882 conn(0x7f2958074cb0 msgr2=0x7f29580750c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:03.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.473+0000 7f295fb52640 1 -- 172.21.15.44:0/3366897882 shutdown_connections 2024-04-22T06:31:03.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.473+0000 7f295fb52640 1 -- 172.21.15.44:0/3366897882 wait complete. 2024-04-22T06:31:03.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.474+0000 7f295fb52640 1 Processor -- start 2024-04-22T06:31:03.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.474+0000 7f295fb52640 1 -- start start 2024-04-22T06:31:03.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.475+0000 7f295fb52640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2958074260 0x7f295814f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:03.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.475+0000 7f295fb52640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2958154f40 0x7f2958157970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:03.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.475+0000 7f295fb52640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2958157eb0 0x7f29580cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:03.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.475+0000 7f295fb52640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f295815e6e0 con 0x7f2958074260 2024-04-22T06:31:03.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.475+0000 7f295fb52640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f295815bcc0 con 0x7f2958154f40 2024-04-22T06:31:03.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.475+0000 7f295fb52640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f295815b050 con 0x7f2958157eb0 2024-04-22T06:31:03.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.475+0000 7f295d8ee640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2958074260 0x7f295814f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:03.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.475+0000 7f295d8ee640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2958074260 0x7f295814f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60532/0 (socket says 172.21.15.44:60532) 2024-04-22T06:31:03.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.475+0000 7f295d8ee640 1 -- 172.21.15.44:0/819445613 learned_addr learned my addr 172.21.15.44:0/819445613 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:03.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.475+0000 7f295d0ed640 1 --2- 172.21.15.44:0/819445613 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2958154f40 0x7f2958157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:03.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.475+0000 7f295e0ef640 1 --2- 172.21.15.44:0/819445613 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2958157eb0 0x7f29580cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:03.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.475+0000 7f295d8ee640 1 -- 172.21.15.44:0/819445613 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2958157eb0 msgr2=0x7f29580cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:03.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.475+0000 7f295d8ee640 1 --2- 172.21.15.44:0/819445613 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2958157eb0 0x7f29580cc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:03.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.475+0000 7f295d8ee640 1 -- 172.21.15.44:0/819445613 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2958154f40 msgr2=0x7f2958157970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:03.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.475+0000 7f295d8ee640 1 --2- 172.21.15.44:0/819445613 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2958154f40 0x7f2958157970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:03.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.475+0000 7f295d8ee640 1 -- 172.21.15.44:0/819445613 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2958150ea0 con 0x7f2958074260 2024-04-22T06:31:03.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.475+0000 7f295e0ef640 1 --2- 172.21.15.44:0/819445613 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2958157eb0 0x7f29580cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:31:03.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.476+0000 7f295d8ee640 1 --2- 172.21.15.44:0/819445613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2958074260 0x7f295814f450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f295000c460 tx=0x7f295000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:03.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.476+0000 7f29567fc640 1 -- 172.21.15.44:0/819445613 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2950016020 con 0x7f2958074260 2024-04-22T06:31:03.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.476+0000 7f29567fc640 1 -- 172.21.15.44:0/819445613 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2950012870 con 0x7f2958074260 2024-04-22T06:31:03.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.476+0000 7f29567fc640 1 -- 172.21.15.44:0/819445613 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2950012c60 con 0x7f2958074260 2024-04-22T06:31:03.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.476+0000 7f295fb52640 1 -- 172.21.15.44:0/819445613 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2958158600 con 0x7f2958074260 2024-04-22T06:31:03.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.476+0000 7f295fb52640 1 -- 172.21.15.44:0/819445613 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29581569f0 con 0x7f2958074260 2024-04-22T06:31:03.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.477+0000 7f295fb52640 1 -- 172.21.15.44:0/819445613 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f295806d590 con 0x7f2958074260 2024-04-22T06:31:03.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.477+0000 7f29567fc640 1 -- 172.21.15.44:0/819445613 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2950019020 con 0x7f2958074260 2024-04-22T06:31:03.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.478+0000 7f29567fc640 1 --2- 172.21.15.44:0/819445613 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f294406c920 0x7f294406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:03.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.478+0000 7f29567fc640 1 -- 172.21.15.44:0/819445613 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f295008b840 con 0x7f2958074260 2024-04-22T06:31:03.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.478+0000 7f295d0ed640 1 --2- 172.21.15.44:0/819445613 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f294406c920 0x7f294406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:03.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.478+0000 7f295d0ed640 1 --2- 172.21.15.44:0/819445613 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f294406c920 0x7f294406ede0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f29581599a0 tx=0x7f294c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:03.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.482+0000 7f29567fc640 1 -- 172.21.15.44:0/819445613 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f295806d590 con 0x7f2958074260 2024-04-22T06:31:03.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.667+0000 7f295fb52640 1 -- 172.21.15.44:0/819445613 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2958156d00 con 0x7f2958074260 2024-04-22T06:31:03.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.668+0000 7f29567fc640 1 -- 172.21.15.44:0/819445613 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12836 (secure 0 0 0) 0x7f2950020020 con 0x7f2958074260 2024-04-22T06:31:03.667 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:31:03.667 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":17,"fsid":"9b0a34ca-403d-4513-8a66-d3749ba559aa","created":"2024-04-22T06:30:42.334581+0000","modified":"2024-04-22T06:30:56.957495+0000","last_up_change":"2024-04-22T06:30:52.942257+0000","last_in_change":"2024-04-22T06:30:44.467727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T06:30:54.560256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2b4a6692-53c9-472d-b2f3-1dd29a6679ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6817","nonce":2220268493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6819","nonce":2220268493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6823","nonce":2220268493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6821","nonce":2220268493}]},"public_addr":"172.21.15.44:6817/2220268493","cluster_addr":"172.21.15.44:6819/2220268493","heartbeat_back_addr":"172.21.15.44:6823/2220268493","heartbeat_front_addr":"172.21.15.44:6821/2220268493","state":["exists","up"]},{"osd":1,"uuid":"70a6370a-7c0a-4d64-a0e4-b2e726b4f291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6809","nonce":1278678977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6811","nonce":1278678977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6815","nonce":1278678977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6813","nonce":1278678977}]},"public_addr":"172.21.15.44:6809/1278678977","cluster_addr":"172.21.15.44:6811/1278678977","heartbeat_back_addr":"172.21.15.44:6815/1278678977","heartbeat_front_addr":"172.21.15.44:6813/1278678977","state":["exists","up"]},{"osd":2,"uuid":"f2e51a62-ca91-470d-bd5a-5710c61deee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6801","nonce":2544087903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6803","nonce":2544087903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6807","nonce":2544087903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6805","nonce":2544087903}]},"public_addr":"172.21.15.44:6801/2544087903","cluster_addr":"172.21.15.44:6803/2544087903","heartbeat_back_addr":"172.21.15.44:6807/2544087903","heartbeat_front_addr":"172.21.15.44:6805/2544087903","state":["exists","up"]},{"osd":3,"uuid":"bc514203-9af9-4f1f-b18e-765bdd5de6e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6825","nonce":1692928109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6827","nonce":1692928109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6831","nonce":1692928109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6829","nonce":1692928109}]},"public_addr":"172.21.15.44:6825/1692928109","cluster_addr":"172.21.15.44:6827/1692928109","heartbeat_back_addr":"172.21.15.44:6831/1692928109","heartbeat_front_addr":"172.21.15.44:6829/1692928109","state":["exists","up"]},{"osd":4,"uuid":"357c439a-6e08-4c19-ac18-b68e6c890497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6820","nonce":3802577427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6824","nonce":3802577427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6831","nonce":3802577427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6828","nonce":3802577427}]},"public_addr":"172.21.15.70:6820/3802577427","cluster_addr":"172.21.15.70:6824/3802577427","heartbeat_back_addr":"172.21.15.70:6831/3802577427","heartbeat_front_addr":"172.21.15.70:6828/3802577427","state":["exists","up"]},{"osd":5,"uuid":"952fb88f-d56e-4305-bfa8-d32674332dfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6801","nonce":664757345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6803","nonce":664757345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6807","nonce":664757345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6805","nonce":664757345}]},"public_addr":"172.21.15.70:6801/664757345","cluster_addr":"172.21.15.70:6803/664757345","heartbeat_back_addr":"172.21.15.70:6807/664757345","heartbeat_front_addr":"172.21.15.70:6805/664757345","state":["exists","up"]},{"osd":6,"uuid":"74399b3d-fac2-45ab-9172-2cd95b02450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6817","nonce":1623253482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6821","nonce":1623253482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6827","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6829","nonce":1623253482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6825","nonce":1623253482}]},"public_addr":"172.21.15.70:6817/1623253482","cluster_addr":"172.21.15.70:6821/1623253482","heartbeat_back_addr":"172.21.15.70:6829/1623253482","heartbeat_front_addr":"172.21.15.70:6825/1623253482","state":["exists","up"]},{"osd":7,"uuid":"8f35508d-6fcc-43ff-8ee7-0498a7ab3b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6809","nonce":1551180300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6811","nonce":1551180300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6815","nonce":1551180300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6813","nonce":1551180300}]},"public_addr":"172.21.15.70:6809/1551180300","cluster_addr":"172.21.15.70:6811/1551180300","heartbeat_back_addr":"172.21.15.70:6815/1551180300","heartbeat_front_addr":"172.21.15.70:6813/1551180300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.612502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.653756+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.521169+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.649955+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.640231+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.686638+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.642199+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.700360+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T06:31:03.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.670+0000 7f295fb52640 1 -- 172.21.15.44:0/819445613 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f294406c920 msgr2=0x7f294406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:03.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.670+0000 7f295fb52640 1 --2- 172.21.15.44:0/819445613 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f294406c920 0x7f294406ede0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f29581599a0 tx=0x7f294c008040 comp rx=0 tx=0).stop 2024-04-22T06:31:03.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.670+0000 7f295fb52640 1 -- 172.21.15.44:0/819445613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2958074260 msgr2=0x7f295814f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:03.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.670+0000 7f295fb52640 1 --2- 172.21.15.44:0/819445613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2958074260 0x7f295814f450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f295000c460 tx=0x7f295000c920 comp rx=0 tx=0).stop 2024-04-22T06:31:03.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.670+0000 7f295fb52640 1 -- 172.21.15.44:0/819445613 shutdown_connections 2024-04-22T06:31:03.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.670+0000 7f295fb52640 1 --2- 172.21.15.44:0/819445613 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f294406c920 0x7f294406ede0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:03.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.670+0000 7f295fb52640 1 --2- 172.21.15.44:0/819445613 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2958157eb0 0x7f29580cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:03.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.670+0000 7f295fb52640 1 --2- 172.21.15.44:0/819445613 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2958154f40 0x7f2958157970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:03.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.670+0000 7f295fb52640 1 --2- 172.21.15.44:0/819445613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2958074260 0x7f295814f450 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:03.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.670+0000 7f295fb52640 1 -- 172.21.15.44:0/819445613 >> 172.21.15.44:0/819445613 conn(0x7f2958074cb0 msgr2=0x7f29580ceb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:03.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.670+0000 7f295fb52640 1 -- 172.21.15.44:0/819445613 shutdown_connections 2024-04-22T06:31:03.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.670+0000 7f295fb52640 1 -- 172.21.15.44:0/819445613 wait complete. 2024-04-22T06:31:03.680 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T06:30:54.560256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T06:31:03.680 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T06:31:03.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.785+0000 7fcd75b04640 1 Processor -- start 2024-04-22T06:31:03.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.785+0000 7fcd75b04640 1 -- start start 2024-04-22T06:31:03.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.786+0000 7fcd75b04640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcd70154f00 0x7fcd700cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:03.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.786+0000 7fcd75b04640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fcd7006d470 0x7fcd700d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:03.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.786+0000 7fcd75b04640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd7015ccf0 con 0x7fcd701552e0 2024-04-22T06:31:03.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.786+0000 7fcd75b04640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fcd700c6740 con 0x7fcd7006d470 2024-04-22T06:31:03.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.786+0000 7fcd75b04640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fcd700c5b30 con 0x7fcd70154f00 2024-04-22T06:31:03.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.786+0000 7fcd6ffff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fcd7006d470 0x7fcd700d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:03.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.786+0000 7fcd6effd640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fcd701552e0 0x7fcd7015c5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:56344/0 (socket says 172.21.15.44:56344) 2024-04-22T06:31:03.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.786+0000 7fcd6ffff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fcd7006d470 0x7fcd700d0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60538/0 (socket says 172.21.15.44:60538) 2024-04-22T06:31:03.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.786+0000 7fcd6effd640 1 -- 172.21.15.44:0/3990326827 learned_addr learned my addr 172.21.15.44:0/3990326827 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:03.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.786+0000 7fcd6f7fe640 1 --2- 172.21.15.44:0/3990326827 >> v2:172.21.15.70:3300/0 conn(0x7fcd70154f00 0x7fcd700cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:03.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.786+0000 7fcd6ffff640 1 -- 172.21.15.44:0/3990326827 >> v2:172.21.15.70:3300/0 conn(0x7fcd70154f00 msgr2=0x7fcd700cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:03.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.786+0000 7fcd6ffff640 1 --2- 172.21.15.44:0/3990326827 >> v2:172.21.15.70:3300/0 conn(0x7fcd70154f00 0x7fcd700cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:03.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.786+0000 7fcd6ffff640 1 -- 172.21.15.44:0/3990326827 >> v1:172.21.15.44:6789/0 conn(0x7fcd701552e0 legacy=0x7fcd7015c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:03.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.786+0000 7fcd6ffff640 1 -- 172.21.15.44:0/3990326827 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd700c5e10 con 0x7fcd7006d470 2024-04-22T06:31:03.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.787+0000 7fcd6ffff640 1 --2- 172.21.15.44:0/3990326827 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd7006d470 0x7fcd700d0f80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcd7006cb60 tx=0x7fcd6802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738642e7990d8c29 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:03.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.787+0000 7fcd6e7fc640 1 -- 172.21.15.44:0/3990326827 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd68039070 con 0x7fcd7006d470 2024-04-22T06:31:03.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.787+0000 7fcd6e7fc640 1 -- 172.21.15.44:0/3990326827 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd6802cbf0 con 0x7fcd7006d470 2024-04-22T06:31:03.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.787+0000 7fcd6e7fc640 1 -- 172.21.15.44:0/3990326827 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd68034440 con 0x7fcd7006d470 2024-04-22T06:31:03.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.787+0000 7fcd75b04640 1 -- 172.21.15.44:0/3990326827 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd7006d470 msgr2=0x7fcd700d0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:03.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.787+0000 7fcd75b04640 1 --2- 172.21.15.44:0/3990326827 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd7006d470 0x7fcd700d0f80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcd7006cb60 tx=0x7fcd6802be40 comp rx=0 tx=0).stop 2024-04-22T06:31:03.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.787+0000 7fcd75b04640 1 -- 172.21.15.44:0/3990326827 shutdown_connections 2024-04-22T06:31:03.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.787+0000 7fcd75b04640 1 --2- 172.21.15.44:0/3990326827 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd7006d470 0x7fcd700d0f80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:03.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.787+0000 7fcd75b04640 1 --2- 172.21.15.44:0/3990326827 >> v2:172.21.15.70:3300/0 conn(0x7fcd70154f00 0x7fcd700cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:03.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.787+0000 7fcd75b04640 1 -- 172.21.15.44:0/3990326827 >> 172.21.15.44:0/3990326827 conn(0x7fcd700733c0 msgr2=0x7fcd700737d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:03.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.787+0000 7fcd75b04640 1 -- 172.21.15.44:0/3990326827 shutdown_connections 2024-04-22T06:31:03.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.788+0000 7fcd75b04640 1 -- 172.21.15.44:0/3990326827 wait complete. 2024-04-22T06:31:03.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.789+0000 7fcd75b04640 1 Processor -- start 2024-04-22T06:31:03.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.789+0000 7fcd75b04640 1 -- start start 2024-04-22T06:31:03.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.789+0000 7fcd75b04640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd7006d470 0x7fcd70150700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:03.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.789+0000 7fcd75b04640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd70154f00 0x7fcd700ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:03.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.790+0000 7fcd75b04640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd701552e0 0x7fcd7015e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:03.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.790+0000 7fcd75b04640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcd700c6bd0 con 0x7fcd701552e0 2024-04-22T06:31:03.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.790+0000 7fcd75b04640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fcd700c63d0 con 0x7fcd7006d470 2024-04-22T06:31:03.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.790+0000 7fcd75b04640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fcd700c6d50 con 0x7fcd70154f00 2024-04-22T06:31:03.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.790+0000 7fcd6ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd701552e0 0x7fcd7015e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:03.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.790+0000 7fcd6ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd701552e0 0x7fcd7015e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60540/0 (socket says 172.21.15.44:60540) 2024-04-22T06:31:03.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.790+0000 7fcd6ffff640 1 -- 172.21.15.44:0/1003220765 learned_addr learned my addr 172.21.15.44:0/1003220765 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:03.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.790+0000 7fcd6ffff640 1 -- 172.21.15.44:0/1003220765 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd70154f00 msgr2=0x7fcd700ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:03.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.790+0000 7fcd6ffff640 1 --2- 172.21.15.44:0/1003220765 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd70154f00 0x7fcd700ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:03.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.790+0000 7fcd6ffff640 1 -- 172.21.15.44:0/1003220765 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd7006d470 msgr2=0x7fcd70150700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:03.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.790+0000 7fcd6f7fe640 1 --2- 172.21.15.44:0/1003220765 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd7006d470 0x7fcd70150700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:03.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.790+0000 7fcd6ffff640 1 --2- 172.21.15.44:0/1003220765 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd7006d470 0x7fcd70150700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:03.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.790+0000 7fcd6ffff640 1 -- 172.21.15.44:0/1003220765 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd7014fb50 con 0x7fcd701552e0 2024-04-22T06:31:03.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.790+0000 7fcd6ffff640 1 --2- 172.21.15.44:0/1003220765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd701552e0 0x7fcd7015e380 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009e80 tx=0x7fcd68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:03.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.790+0000 7fcd57fff640 1 -- 172.21.15.44:0/1003220765 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd68039050 con 0x7fcd701552e0 2024-04-22T06:31:03.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.790+0000 7fcd57fff640 1 -- 172.21.15.44:0/1003220765 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd68045070 con 0x7fcd701552e0 2024-04-22T06:31:03.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.790+0000 7fcd75b04640 1 -- 172.21.15.44:0/1003220765 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd700ceef0 con 0x7fcd701552e0 2024-04-22T06:31:03.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.790+0000 7fcd57fff640 1 -- 172.21.15.44:0/1003220765 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6803e830 con 0x7fcd701552e0 2024-04-22T06:31:03.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.790+0000 7fcd75b04640 1 -- 172.21.15.44:0/1003220765 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd701f4730 con 0x7fcd701552e0 2024-04-22T06:31:03.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.791+0000 7fcd75b04640 1 -- 172.21.15.44:0/1003220765 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd70068cc0 con 0x7fcd701552e0 2024-04-22T06:31:03.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.792+0000 7fcd57fff640 1 -- 172.21.15.44:0/1003220765 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcd6803e9d0 con 0x7fcd701552e0 2024-04-22T06:31:03.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.792+0000 7fcd57fff640 1 --2- 172.21.15.44:0/1003220765 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fcd5c06c8f0 0x7fcd5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:03.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.792+0000 7fcd57fff640 1 -- 172.21.15.44:0/1003220765 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fcd680ae620 con 0x7fcd701552e0 2024-04-22T06:31:03.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.793+0000 7fcd6f7fe640 1 --2- 172.21.15.44:0/1003220765 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fcd5c06c8f0 0x7fcd5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:03.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.793+0000 7fcd6f7fe640 1 --2- 172.21.15.44:0/1003220765 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fcd5c06c8f0 0x7fcd5c06edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcd60006fd0 tx=0x7fcd60008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:03.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.797+0000 7fcd57fff640 1 -- 172.21.15.44:0/1003220765 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcd6807cc70 con 0x7fcd701552e0 2024-04-22T06:31:03.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.983+0000 7fcd75b04640 1 -- 172.21.15.44:0/1003220765 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcd700caee0 con 0x7fcd701552e0 2024-04-22T06:31:03.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.984+0000 7fcd57fff640 1 -- 172.21.15.44:0/1003220765 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) ==== 93+0+10 (secure 0 0 0) 0x7fcd6807c570 con 0x7fcd701552e0 2024-04-22T06:31:03.983 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-04-22T06:31:03.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.986+0000 7fcd75b04640 1 -- 172.21.15.44:0/1003220765 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fcd5c06c8f0 msgr2=0x7fcd5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:03.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.986+0000 7fcd75b04640 1 --2- 172.21.15.44:0/1003220765 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fcd5c06c8f0 0x7fcd5c06edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcd60006fd0 tx=0x7fcd60008040 comp rx=0 tx=0).stop 2024-04-22T06:31:03.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.986+0000 7fcd75b04640 1 -- 172.21.15.44:0/1003220765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd701552e0 msgr2=0x7fcd7015e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:03.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.986+0000 7fcd75b04640 1 --2- 172.21.15.44:0/1003220765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd701552e0 0x7fcd7015e380 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009e80 tx=0x7fcd68004a40 comp rx=0 tx=0).stop 2024-04-22T06:31:03.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.986+0000 7fcd75b04640 1 -- 172.21.15.44:0/1003220765 shutdown_connections 2024-04-22T06:31:03.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.986+0000 7fcd75b04640 1 --2- 172.21.15.44:0/1003220765 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fcd5c06c8f0 0x7fcd5c06edb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:03.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.986+0000 7fcd75b04640 1 --2- 172.21.15.44:0/1003220765 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd70154f00 0x7fcd700ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:03.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.986+0000 7fcd75b04640 1 --2- 172.21.15.44:0/1003220765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd701552e0 0x7fcd7015e380 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:03.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.986+0000 7fcd75b04640 1 --2- 172.21.15.44:0/1003220765 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd7006d470 0x7fcd70150700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:03.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.986+0000 7fcd75b04640 1 -- 172.21.15.44:0/1003220765 >> 172.21.15.44:0/1003220765 conn(0x7fcd700733c0 msgr2=0x7fcd70152410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:03.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.986+0000 7fcd75b04640 1 -- 172.21.15.44:0/1003220765 shutdown_connections 2024-04-22T06:31:03.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:03.986+0000 7fcd75b04640 1 -- 172.21.15.44:0/1003220765 wait complete. 2024-04-22T06:31:03.995 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-22T06:31:04.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.097+0000 7f1199b67640 1 Processor -- start 2024-04-22T06:31:04.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.097+0000 7f1199b67640 1 -- start start 2024-04-22T06:31:04.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.097+0000 7f1199b67640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1194154f00 0x7f11940cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:04.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.097+0000 7f1199b67640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f119406d470 0x7f11940d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:04.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.097+0000 7f1199b67640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f119415ccf0 con 0x7f11941552e0 2024-04-22T06:31:04.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.097+0000 7f1199b67640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f11940c6740 con 0x7f119406d470 2024-04-22T06:31:04.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.098+0000 7f1199b67640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f11940c5b30 con 0x7f1194154f00 2024-04-22T06:31:04.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.098+0000 7f1193fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f119406d470 0x7f11940d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:04.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.098+0000 7f1193fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f119406d470 0x7f11940d0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60548/0 (socket says 172.21.15.44:60548) 2024-04-22T06:31:04.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.098+0000 7f1193fff640 1 -- 172.21.15.44:0/929490956 learned_addr learned my addr 172.21.15.44:0/929490956 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:04.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.098+0000 7f11937fe640 1 --2- 172.21.15.44:0/929490956 >> v2:172.21.15.70:3300/0 conn(0x7f1194154f00 0x7f11940cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:04.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.098+0000 7f1193fff640 1 -- 172.21.15.44:0/929490956 >> v2:172.21.15.70:3300/0 conn(0x7f1194154f00 msgr2=0x7f11940cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:04.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.098+0000 7f1193fff640 1 --2- 172.21.15.44:0/929490956 >> v2:172.21.15.70:3300/0 conn(0x7f1194154f00 0x7f11940cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:04.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.098+0000 7f1193fff640 1 -- 172.21.15.44:0/929490956 >> v1:172.21.15.70:6789/0 conn(0x7f11941552e0 legacy=0x7f119415c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:04.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.098+0000 7f1193fff640 1 -- 172.21.15.44:0/929490956 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11940c5e10 con 0x7f119406d470 2024-04-22T06:31:04.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.099+0000 7f1193fff640 1 --2- 172.21.15.44:0/929490956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f119406d470 0x7f11940d0f80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f119406cb60 tx=0x7f118c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf4afae54a4e98c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:04.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.099+0000 7f11927fc640 1 -- 172.21.15.44:0/929490956 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118c02c9f0 con 0x7f119406d470 2024-04-22T06:31:04.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.099+0000 7f11927fc640 1 -- 172.21.15.44:0/929490956 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f118c02cb90 con 0x7f119406d470 2024-04-22T06:31:04.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.099+0000 7f11927fc640 1 -- 172.21.15.44:0/929490956 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118c0343f0 con 0x7f119406d470 2024-04-22T06:31:04.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.099+0000 7f1199b67640 1 -- 172.21.15.44:0/929490956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f119406d470 msgr2=0x7f11940d0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:04.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.099+0000 7f1199b67640 1 --2- 172.21.15.44:0/929490956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f119406d470 0x7f11940d0f80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f119406cb60 tx=0x7f118c02be50 comp rx=0 tx=0).stop 2024-04-22T06:31:04.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.099+0000 7f1199b67640 1 -- 172.21.15.44:0/929490956 shutdown_connections 2024-04-22T06:31:04.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.099+0000 7f1199b67640 1 --2- 172.21.15.44:0/929490956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f119406d470 0x7f11940d0f80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:04.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.099+0000 7f1199b67640 1 --2- 172.21.15.44:0/929490956 >> v2:172.21.15.70:3300/0 conn(0x7f1194154f00 0x7f11940cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:04.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.099+0000 7f1199b67640 1 -- 172.21.15.44:0/929490956 >> 172.21.15.44:0/929490956 conn(0x7f11940733c0 msgr2=0x7f11940737d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:04.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.099+0000 7f1199b67640 1 -- 172.21.15.44:0/929490956 shutdown_connections 2024-04-22T06:31:04.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.099+0000 7f1199b67640 1 -- 172.21.15.44:0/929490956 wait complete. 2024-04-22T06:31:04.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.101+0000 7f1199b67640 1 Processor -- start 2024-04-22T06:31:04.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.101+0000 7f1199b67640 1 -- start start 2024-04-22T06:31:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.101+0000 7f1199b67640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f119406d470 0x7f1194159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.101+0000 7f1199b67640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1194154f00 0x7f119415a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.101+0000 7f1199b67640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11941552e0 0x7f11940cc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.101+0000 7f1199b67640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f11940c6bd0 con 0x7f11941552e0 2024-04-22T06:31:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.101+0000 7f1199b67640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f11940c63d0 con 0x7f119406d470 2024-04-22T06:31:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.101+0000 7f1199b67640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f11940c6d50 con 0x7f1194154f00 2024-04-22T06:31:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.101+0000 7f1193fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11941552e0 0x7f11940cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.102+0000 7f1193fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11941552e0 0x7f11940cc860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60556/0 (socket says 172.21.15.44:60556) 2024-04-22T06:31:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.102+0000 7f1193fff640 1 -- 172.21.15.44:0/492663726 learned_addr learned my addr 172.21.15.44:0/492663726 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.102+0000 7f11937fe640 1 --2- 172.21.15.44:0/492663726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f119406d470 0x7f1194159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.102+0000 7f1192ffd640 1 --2- 172.21.15.44:0/492663726 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1194154f00 0x7f119415a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.102+0000 7f1193fff640 1 -- 172.21.15.44:0/492663726 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1194154f00 msgr2=0x7f119415a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.102+0000 7f1193fff640 1 --2- 172.21.15.44:0/492663726 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1194154f00 0x7f119415a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.102+0000 7f1193fff640 1 -- 172.21.15.44:0/492663726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f119406d470 msgr2=0x7f1194159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.102+0000 7f1193fff640 1 --2- 172.21.15.44:0/492663726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f119406d470 0x7f1194159bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.102+0000 7f1193fff640 1 -- 172.21.15.44:0/492663726 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11941f1a10 con 0x7f11941552e0 2024-04-22T06:31:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.102+0000 7f11937fe640 1 --2- 172.21.15.44:0/492663726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f119406d470 0x7f1194159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:31:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.102+0000 7f1192ffd640 1 --2- 172.21.15.44:0/492663726 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1194154f00 0x7f119415a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:31:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.102+0000 7f1193fff640 1 --2- 172.21.15.44:0/492663726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11941552e0 0x7f11940cc860 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f118c009f40 tx=0x7f118c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:04.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.102+0000 7f118bfff640 1 -- 172.21.15.44:0/492663726 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118c043020 con 0x7f11941552e0 2024-04-22T06:31:04.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.102+0000 7f118bfff640 1 -- 172.21.15.44:0/492663726 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f118c03e450 con 0x7f11941552e0 2024-04-22T06:31:04.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.102+0000 7f118bfff640 1 -- 172.21.15.44:0/492663726 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118c009510 con 0x7f11941552e0 2024-04-22T06:31:04.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.102+0000 7f1199b67640 1 -- 172.21.15.44:0/492663726 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11941f1cf0 con 0x7f11941552e0 2024-04-22T06:31:04.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.102+0000 7f1199b67640 1 -- 172.21.15.44:0/492663726 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11940cc060 con 0x7f11941552e0 2024-04-22T06:31:04.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.103+0000 7f1199b67640 1 -- 172.21.15.44:0/492663726 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11940caee0 con 0x7f11941552e0 2024-04-22T06:31:04.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.104+0000 7f118bfff640 1 -- 172.21.15.44:0/492663726 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f118c046020 con 0x7f11941552e0 2024-04-22T06:31:04.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.104+0000 7f118bfff640 1 --2- 172.21.15.44:0/492663726 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f117406c920 0x7f117406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:04.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.104+0000 7f118bfff640 1 -- 172.21.15.44:0/492663726 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f118c0add20 con 0x7f11941552e0 2024-04-22T06:31:04.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.105+0000 7f11937fe640 1 --2- 172.21.15.44:0/492663726 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f117406c920 0x7f117406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:04.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.105+0000 7f11937fe640 1 --2- 172.21.15.44:0/492663726 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f117406c920 0x7f117406ede0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f117c006fd0 tx=0x7f117c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:04.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.109+0000 7f118bfff640 1 -- 172.21.15.44:0/492663726 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f118c04d070 con 0x7f11941552e0 2024-04-22T06:31:04.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.294+0000 7f1199b67640 1 -- 172.21.15.44:0/492663726 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f11940b5e60 con 0x7f11941552e0 2024-04-22T06:31:04.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.294+0000 7f118bfff640 1 -- 172.21.15.44:0/492663726 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7f118c07c530 con 0x7f11941552e0 2024-04-22T06:31:04.293 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:31:04.293 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-04-22T06:31:04.293 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-22T06:31:04.294 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2024-04-22T06:31:04.294 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2024-04-22T06:31:04.294 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-22T06:31:04.294 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-04-22T06:31:04.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.296+0000 7f1199b67640 1 -- 172.21.15.44:0/492663726 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f117406c920 msgr2=0x7f117406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:04.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.296+0000 7f1199b67640 1 --2- 172.21.15.44:0/492663726 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f117406c920 0x7f117406ede0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f117c006fd0 tx=0x7f117c008040 comp rx=0 tx=0).stop 2024-04-22T06:31:04.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.296+0000 7f1199b67640 1 -- 172.21.15.44:0/492663726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11941552e0 msgr2=0x7f11940cc860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:04.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.296+0000 7f1199b67640 1 --2- 172.21.15.44:0/492663726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11941552e0 0x7f11940cc860 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f118c009f40 tx=0x7f118c004a40 comp rx=0 tx=0).stop 2024-04-22T06:31:04.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.296+0000 7f1199b67640 1 -- 172.21.15.44:0/492663726 shutdown_connections 2024-04-22T06:31:04.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.296+0000 7f1199b67640 1 --2- 172.21.15.44:0/492663726 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f117406c920 0x7f117406ede0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:04.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.296+0000 7f1199b67640 1 --2- 172.21.15.44:0/492663726 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1194154f00 0x7f119415a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:04.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.296+0000 7f1199b67640 1 --2- 172.21.15.44:0/492663726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11941552e0 0x7f11940cc860 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:04.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.296+0000 7f1199b67640 1 --2- 172.21.15.44:0/492663726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f119406d470 0x7f1194159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:04.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.296+0000 7f1199b67640 1 -- 172.21.15.44:0/492663726 >> 172.21.15.44:0/492663726 conn(0x7f11940733c0 msgr2=0x7f11940d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:04.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.296+0000 7f1199b67640 1 -- 172.21.15.44:0/492663726 shutdown_connections 2024-04-22T06:31:04.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.296+0000 7f1199b67640 1 -- 172.21.15.44:0/492663726 wait complete. 2024-04-22T06:31:04.305 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-22T06:31:04.306 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-22T06:31:04.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.411+0000 7fc064b9f640 1 Processor -- start 2024-04-22T06:31:04.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.411+0000 7fc064b9f640 1 -- start start 2024-04-22T06:31:04.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.412+0000 7fc064b9f640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc0601550c0 0x7fc06015aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:04.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.412+0000 7fc064b9f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc0600c5af0 0x7fc06015ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:04.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.412+0000 7fc064b9f640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc06015b590 con 0x7fc0600cb430 2024-04-22T06:31:04.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.412+0000 7fc064b9f640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fc06015b870 con 0x7fc0601550c0 2024-04-22T06:31:04.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.412+0000 7fc064b9f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fc06015bb50 con 0x7fc0600c5af0 2024-04-22T06:31:04.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.412+0000 7fc05f7fe640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc0601550c0 0x7fc06015aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:04.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.412+0000 7fc05ffff640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fc0600cb430 0x7fc0600cc1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:56346/0 (socket says 172.21.15.44:56346) 2024-04-22T06:31:04.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.412+0000 7fc05ffff640 1 -- 172.21.15.44:0/4264451398 learned_addr learned my addr 172.21.15.44:0/4264451398 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:04.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.412+0000 7fc05effd640 1 --2- 172.21.15.44:0/4264451398 >> v2:172.21.15.70:3300/0 conn(0x7fc0600c5af0 0x7fc06015ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:04.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.413+0000 7fc05f7fe640 1 -- 172.21.15.44:0/4264451398 >> v2:172.21.15.70:3300/0 conn(0x7fc0600c5af0 msgr2=0x7fc06015ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:04.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.413+0000 7fc05f7fe640 1 --2- 172.21.15.44:0/4264451398 >> v2:172.21.15.70:3300/0 conn(0x7fc0600c5af0 0x7fc06015ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:04.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.413+0000 7fc05f7fe640 1 -- 172.21.15.44:0/4264451398 >> v1:172.21.15.44:6789/0 conn(0x7fc0600cb430 legacy=0x7fc0600cc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:04.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.413+0000 7fc05f7fe640 1 -- 172.21.15.44:0/4264451398 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc06015e600 con 0x7fc0601550c0 2024-04-22T06:31:04.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.413+0000 7fc05e7fc640 1 -- 172.21.15.44:0/4264451398 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2005532925 0 0) 0x7fc06015b590 con 0x7fc0600cb430 2024-04-22T06:31:04.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.413+0000 7fc05effd640 1 --2- 172.21.15.44:0/4264451398 >> v2:172.21.15.70:3300/0 conn(0x7fc0600c5af0 0x7fc06015ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:31:04.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.413+0000 7fc05f7fe640 1 --2- 172.21.15.44:0/4264451398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0601550c0 0x7fc06015aee0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc0600b4660 tx=0x7fc04802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4ffeb70e777fa94 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:04.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.413+0000 7fc05e7fc640 1 -- 172.21.15.44:0/4264451398 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc048035020 con 0x7fc0601550c0 2024-04-22T06:31:04.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.413+0000 7fc05e7fc640 1 -- 172.21.15.44:0/4264451398 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc048031830 con 0x7fc0601550c0 2024-04-22T06:31:04.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.413+0000 7fc05e7fc640 1 -- 172.21.15.44:0/4264451398 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc048031c00 con 0x7fc0601550c0 2024-04-22T06:31:04.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.414+0000 7fc064b9f640 1 -- 172.21.15.44:0/4264451398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0601550c0 msgr2=0x7fc06015aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:04.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.414+0000 7fc064b9f640 1 --2- 172.21.15.44:0/4264451398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0601550c0 0x7fc06015aee0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc0600b4660 tx=0x7fc04802b8d0 comp rx=0 tx=0).stop 2024-04-22T06:31:04.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.414+0000 7fc064b9f640 1 -- 172.21.15.44:0/4264451398 shutdown_connections 2024-04-22T06:31:04.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.414+0000 7fc064b9f640 1 --2- 172.21.15.44:0/4264451398 >> v2:172.21.15.70:3300/0 conn(0x7fc0600c5af0 0x7fc06015ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:04.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.414+0000 7fc064b9f640 1 --2- 172.21.15.44:0/4264451398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0601550c0 0x7fc06015aee0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:04.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.414+0000 7fc064b9f640 1 -- 172.21.15.44:0/4264451398 >> 172.21.15.44:0/4264451398 conn(0x7fc06006d350 msgr2=0x7fc060074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:04.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.414+0000 7fc064b9f640 1 -- 172.21.15.44:0/4264451398 shutdown_connections 2024-04-22T06:31:04.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.414+0000 7fc064b9f640 1 -- 172.21.15.44:0/4264451398 wait complete. 2024-04-22T06:31:04.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.415+0000 7fc064b9f640 1 Processor -- start 2024-04-22T06:31:04.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.415+0000 7fc064b9f640 1 -- start start 2024-04-22T06:31:04.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.416+0000 7fc064b9f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0600c5af0 0x7fc0601f1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:04.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.416+0000 7fc064b9f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0600cb430 0x7fc0600ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:04.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.416+0000 7fc064b9f640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc0601550c0 0x7fc06014afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:04.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.416+0000 7fc064b9f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc0600ccdd0 con 0x7fc0600cb430 2024-04-22T06:31:04.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.416+0000 7fc064b9f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc06015f340 con 0x7fc0600c5af0 2024-04-22T06:31:04.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.416+0000 7fc064b9f640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc0601554a0 con 0x7fc0601550c0 2024-04-22T06:31:04.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.416+0000 7fc05effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0600cb430 0x7fc0600ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:04.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.416+0000 7fc05effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0600cb430 0x7fc0600ca2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60580/0 (socket says 172.21.15.44:60580) 2024-04-22T06:31:04.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.416+0000 7fc05effd640 1 -- 172.21.15.44:0/2705515842 learned_addr learned my addr 172.21.15.44:0/2705515842 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:04.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.416+0000 7fc05ffff640 1 --2- 172.21.15.44:0/2705515842 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc0601550c0 0x7fc06014afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:04.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.416+0000 7fc05f7fe640 1 --2- 172.21.15.44:0/2705515842 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0600c5af0 0x7fc0601f1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:04.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.417+0000 7fc05effd640 1 -- 172.21.15.44:0/2705515842 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc0601550c0 msgr2=0x7fc06014afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:04.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.417+0000 7fc05effd640 1 --2- 172.21.15.44:0/2705515842 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc0601550c0 0x7fc06014afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:04.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.417+0000 7fc05effd640 1 -- 172.21.15.44:0/2705515842 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0600c5af0 msgr2=0x7fc0601f1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:04.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.417+0000 7fc05effd640 1 --2- 172.21.15.44:0/2705515842 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0600c5af0 0x7fc0601f1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:04.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.417+0000 7fc05effd640 1 -- 172.21.15.44:0/2705515842 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0600c92c0 con 0x7fc0600cb430 2024-04-22T06:31:04.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.417+0000 7fc05effd640 1 --2- 172.21.15.44:0/2705515842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0600cb430 0x7fc0600ca2d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc05000c7f0 tx=0x7fc05000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:04.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.417+0000 7fc057fff640 1 -- 172.21.15.44:0/2705515842 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc050015070 con 0x7fc0600cb430 2024-04-22T06:31:04.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.417+0000 7fc057fff640 1 -- 172.21.15.44:0/2705515842 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc050007a30 con 0x7fc0600cb430 2024-04-22T06:31:04.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.417+0000 7fc064b9f640 1 -- 172.21.15.44:0/2705515842 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0600c95a0 con 0x7fc0600cb430 2024-04-22T06:31:04.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.417+0000 7fc057fff640 1 -- 172.21.15.44:0/2705515842 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc050007e00 con 0x7fc0600cb430 2024-04-22T06:31:04.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.417+0000 7fc064b9f640 1 -- 172.21.15.44:0/2705515842 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0601f0d10 con 0x7fc0600cb430 2024-04-22T06:31:04.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.418+0000 7fc064b9f640 1 -- 172.21.15.44:0/2705515842 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0600761b0 con 0x7fc0600cb430 2024-04-22T06:31:04.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.418+0000 7fc057fff640 1 -- 172.21.15.44:0/2705515842 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc050018070 con 0x7fc0600cb430 2024-04-22T06:31:04.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.419+0000 7fc057fff640 1 --2- 172.21.15.44:0/2705515842 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fc04006c8f0 0x7fc04006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:04.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.419+0000 7fc057fff640 1 -- 172.21.15.44:0/2705515842 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fc05008b6c0 con 0x7fc0600cb430 2024-04-22T06:31:04.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.419+0000 7fc05f7fe640 1 --2- 172.21.15.44:0/2705515842 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fc04006c8f0 0x7fc04006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:04.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.419+0000 7fc05f7fe640 1 --2- 172.21.15.44:0/2705515842 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fc04006c8f0 0x7fc04006edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc048036000 tx=0x7fc04803c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:04.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.423+0000 7fc057fff640 1 -- 172.21.15.44:0/2705515842 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc0500597f0 con 0x7fc0600cb430 2024-04-22T06:31:04.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.608+0000 7fc064b9f640 1 -- 172.21.15.44:0/2705515842 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc060153280 con 0x7fc0600cb430 2024-04-22T06:31:04.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.977+0000 7fc057fff640 1 -- 172.21.15.44:0/2705515842 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7fc05005daf0 con 0x7fc0600cb430 2024-04-22T06:31:04.976 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' created 2024-04-22T06:31:04.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.980+0000 7fc064b9f640 1 -- 172.21.15.44:0/2705515842 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fc04006c8f0 msgr2=0x7fc04006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:04.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.980+0000 7fc064b9f640 1 --2- 172.21.15.44:0/2705515842 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fc04006c8f0 0x7fc04006edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc048036000 tx=0x7fc04803c040 comp rx=0 tx=0).stop 2024-04-22T06:31:04.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.980+0000 7fc064b9f640 1 -- 172.21.15.44:0/2705515842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0600cb430 msgr2=0x7fc0600ca2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:04.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.980+0000 7fc064b9f640 1 --2- 172.21.15.44:0/2705515842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0600cb430 0x7fc0600ca2d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc05000c7f0 tx=0x7fc05000cbf0 comp rx=0 tx=0).stop 2024-04-22T06:31:04.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.980+0000 7fc064b9f640 1 -- 172.21.15.44:0/2705515842 shutdown_connections 2024-04-22T06:31:04.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.980+0000 7fc064b9f640 1 --2- 172.21.15.44:0/2705515842 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fc04006c8f0 0x7fc04006edb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:04.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.980+0000 7fc064b9f640 1 --2- 172.21.15.44:0/2705515842 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc0601550c0 0x7fc06014afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:04.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.980+0000 7fc064b9f640 1 --2- 172.21.15.44:0/2705515842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0600cb430 0x7fc0600ca2d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:04.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.980+0000 7fc064b9f640 1 --2- 172.21.15.44:0/2705515842 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0600c5af0 0x7fc0601f1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:04.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.980+0000 7fc064b9f640 1 -- 172.21.15.44:0/2705515842 >> 172.21.15.44:0/2705515842 conn(0x7fc06006d350 msgr2=0x7fc060072bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:04.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.980+0000 7fc064b9f640 1 -- 172.21.15.44:0/2705515842 shutdown_connections 2024-04-22T06:31:04.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:04.980+0000 7fc064b9f640 1 -- 172.21.15.44:0/2705515842 wait complete. 2024-04-22T06:31:04.997 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-22T06:31:05.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.108+0000 7f7aaad52640 1 Processor -- start 2024-04-22T06:31:05.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.108+0000 7f7aaad52640 1 -- start start 2024-04-22T06:31:05.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.109+0000 7f7aaad52640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f7aa4154f00 0x7f7aa40cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:05.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.109+0000 7f7aaad52640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7aa41552e0 0x7f7aa415c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:05.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.109+0000 7f7aaad52640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7aa40c5b30 con 0x7f7aa406d470 2024-04-22T06:31:05.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.109+0000 7f7aaad52640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f7aa40c6740 con 0x7f7aa4154f00 2024-04-22T06:31:05.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.109+0000 7f7aaad52640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7aa40c5e10 con 0x7f7aa41552e0 2024-04-22T06:31:05.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.109+0000 7f7aa8aee640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f7aa4154f00 0x7f7aa40cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:05.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.109+0000 7f7aa92ef640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f7aa406d470 0x7f7aa40d0f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:56354/0 (socket says 172.21.15.44:56354) 2024-04-22T06:31:05.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.109+0000 7f7aa92ef640 1 -- 172.21.15.44:0/4197129361 learned_addr learned my addr 172.21.15.44:0/4197129361 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:05.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.109+0000 7f7aa3fff640 1 --2- 172.21.15.44:0/4197129361 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7aa41552e0 0x7f7aa415c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:05.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.109+0000 7f7aa8aee640 1 -- 172.21.15.44:0/4197129361 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7aa41552e0 msgr2=0x7f7aa415c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:05.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.109+0000 7f7aa8aee640 1 --2- 172.21.15.44:0/4197129361 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7aa41552e0 0x7f7aa415c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:05.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.109+0000 7f7aa8aee640 1 -- 172.21.15.44:0/4197129361 >> v1:172.21.15.44:6789/0 conn(0x7f7aa406d470 legacy=0x7f7aa40d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:05.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.110+0000 7f7aa8aee640 1 -- 172.21.15.44:0/4197129361 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aa40c60f0 con 0x7f7aa4154f00 2024-04-22T06:31:05.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.109+0000 7f7aa37fe640 1 -- 172.21.15.44:0/4197129361 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1916885167 0 0) 0x7f7aa40c5b30 con 0x7f7aa406d470 2024-04-22T06:31:05.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.110+0000 7f7aa8aee640 1 --2- 172.21.15.44:0/4197129361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7aa4154f00 0x7f7aa40cb430 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c0097a0 tx=0x7f7a9c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0f13635f4b477c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:05.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.110+0000 7f7aa37fe640 1 -- 172.21.15.44:0/4197129361 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c035020 con 0x7f7aa4154f00 2024-04-22T06:31:05.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.110+0000 7f7aa37fe640 1 -- 172.21.15.44:0/4197129361 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a9c031830 con 0x7f7aa4154f00 2024-04-22T06:31:05.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.110+0000 7f7aa37fe640 1 -- 172.21.15.44:0/4197129361 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c031c00 con 0x7f7aa4154f00 2024-04-22T06:31:05.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.111+0000 7f7aaad52640 1 -- 172.21.15.44:0/4197129361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7aa4154f00 msgr2=0x7f7aa40cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:05.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.111+0000 7f7aaad52640 1 --2- 172.21.15.44:0/4197129361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7aa4154f00 0x7f7aa40cb430 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c0097a0 tx=0x7f7a9c02b8d0 comp rx=0 tx=0).stop 2024-04-22T06:31:05.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.111+0000 7f7aaad52640 1 -- 172.21.15.44:0/4197129361 shutdown_connections 2024-04-22T06:31:05.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.111+0000 7f7aaad52640 1 --2- 172.21.15.44:0/4197129361 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7aa41552e0 0x7f7aa415c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:05.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.111+0000 7f7aaad52640 1 --2- 172.21.15.44:0/4197129361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7aa4154f00 0x7f7aa40cb430 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:05.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.111+0000 7f7aaad52640 1 -- 172.21.15.44:0/4197129361 >> 172.21.15.44:0/4197129361 conn(0x7f7aa40733c0 msgr2=0x7f7aa40737d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:05.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.111+0000 7f7aaad52640 1 -- 172.21.15.44:0/4197129361 shutdown_connections 2024-04-22T06:31:05.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.111+0000 7f7aaad52640 1 -- 172.21.15.44:0/4197129361 wait complete. 2024-04-22T06:31:05.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.112+0000 7f7aaad52640 1 Processor -- start 2024-04-22T06:31:05.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.112+0000 7f7aaad52640 1 -- start start 2024-04-22T06:31:05.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.113+0000 7f7aaad52640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7aa406d470 0x7f7aa414a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:05.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.113+0000 7f7aaad52640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7aa4154f00 0x7f7aa414c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:05.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.113+0000 7f7aaad52640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7aa41552e0 0x7f7aa414ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:05.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.113+0000 7f7aaad52640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa40c6d50 con 0x7f7aa41552e0 2024-04-22T06:31:05.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.113+0000 7f7aaad52640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa40c6bd0 con 0x7f7aa4154f00 2024-04-22T06:31:05.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.113+0000 7f7aaad52640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7aa40d1690 con 0x7f7aa406d470 2024-04-22T06:31:05.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.113+0000 7f7aa92ef640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7aa41552e0 0x7f7aa414ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:05.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.113+0000 7f7aa92ef640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7aa41552e0 0x7f7aa414ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60596/0 (socket says 172.21.15.44:60596) 2024-04-22T06:31:05.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.113+0000 7f7aa92ef640 1 -- 172.21.15.44:0/49292861 learned_addr learned my addr 172.21.15.44:0/49292861 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:05.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.113+0000 7f7aa8aee640 1 --2- 172.21.15.44:0/49292861 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7aa406d470 0x7f7aa414a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:05.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.113+0000 7f7aa3fff640 1 --2- 172.21.15.44:0/49292861 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7aa4154f00 0x7f7aa414c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:05.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.113+0000 7f7aa92ef640 1 -- 172.21.15.44:0/49292861 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7aa406d470 msgr2=0x7f7aa414a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:05.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.113+0000 7f7aa92ef640 1 --2- 172.21.15.44:0/49292861 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7aa406d470 0x7f7aa414a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:05.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.113+0000 7f7aa92ef640 1 -- 172.21.15.44:0/49292861 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7aa4154f00 msgr2=0x7f7aa414c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:05.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.113+0000 7f7aa92ef640 1 --2- 172.21.15.44:0/49292861 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7aa4154f00 0x7f7aa414c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:05.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.113+0000 7f7aa92ef640 1 -- 172.21.15.44:0/49292861 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aa41f4f10 con 0x7f7aa41552e0 2024-04-22T06:31:05.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.114+0000 7f7aa92ef640 1 --2- 172.21.15.44:0/49292861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7aa41552e0 0x7f7aa414ca20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c00c480 tx=0x7f7a8c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:05.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.114+0000 7f7aa17fa640 1 -- 172.21.15.44:0/49292861 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c018070 con 0x7f7aa41552e0 2024-04-22T06:31:05.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.114+0000 7f7aa17fa640 1 -- 172.21.15.44:0/49292861 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a8c007500 con 0x7f7aa41552e0 2024-04-22T06:31:05.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.114+0000 7f7aa17fa640 1 -- 172.21.15.44:0/49292861 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c004a20 con 0x7f7aa41552e0 2024-04-22T06:31:05.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.114+0000 7f7aaad52640 1 -- 172.21.15.44:0/49292861 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aa41f0c60 con 0x7f7aa41552e0 2024-04-22T06:31:05.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.114+0000 7f7aaad52640 1 -- 172.21.15.44:0/49292861 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aa41f1220 con 0x7f7aa41552e0 2024-04-22T06:31:05.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.115+0000 7f7aaad52640 1 -- 172.21.15.44:0/49292861 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7aa4076220 con 0x7f7aa41552e0 2024-04-22T06:31:05.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.115+0000 7f7aa17fa640 1 -- 172.21.15.44:0/49292861 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7a8c01b070 con 0x7f7aa41552e0 2024-04-22T06:31:05.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.116+0000 7f7aa17fa640 1 --2- 172.21.15.44:0/49292861 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f7a8406c8f0 0x7f7a8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:05.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.116+0000 7f7aa17fa640 1 -- 172.21.15.44:0/49292861 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7f7a8c08ad50 con 0x7f7aa41552e0 2024-04-22T06:31:05.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.116+0000 7f7aa8aee640 1 --2- 172.21.15.44:0/49292861 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f7a8406c8f0 0x7f7a8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:05.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.116+0000 7f7aa8aee640 1 --2- 172.21.15.44:0/49292861 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f7a8406c8f0 0x7f7a8406edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c036000 tx=0x7f7a9c008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:05.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.120+0000 7f7aa17fa640 1 -- 172.21.15.44:0/49292861 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7aa4076220 con 0x7f7aa41552e0 2024-04-22T06:31:05.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.308+0000 7f7aaad52640 1 -- 172.21.15.44:0/49292861 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f7aa4068cc0 con 0x7f7aa41552e0 2024-04-22T06:31:05.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.980+0000 7f7aa17fa640 1 -- 172.21.15.44:0/49292861 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) ==== 171+0+0 (secure 0 0 0) 0x7f7a8c0592c0 con 0x7f7aa41552e0 2024-04-22T06:31:05.979 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' created 2024-04-22T06:31:05.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.983+0000 7f7aaad52640 1 -- 172.21.15.44:0/49292861 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f7a8406c8f0 msgr2=0x7f7a8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:05.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.983+0000 7f7aaad52640 1 --2- 172.21.15.44:0/49292861 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f7a8406c8f0 0x7f7a8406edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c036000 tx=0x7f7a9c008030 comp rx=0 tx=0).stop 2024-04-22T06:31:05.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.983+0000 7f7aaad52640 1 -- 172.21.15.44:0/49292861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7aa41552e0 msgr2=0x7f7aa414ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:05.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.983+0000 7f7aaad52640 1 --2- 172.21.15.44:0/49292861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7aa41552e0 0x7f7aa414ca20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c00c480 tx=0x7f7a8c00c940 comp rx=0 tx=0).stop 2024-04-22T06:31:05.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.983+0000 7f7aaad52640 1 -- 172.21.15.44:0/49292861 shutdown_connections 2024-04-22T06:31:05.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.983+0000 7f7aaad52640 1 --2- 172.21.15.44:0/49292861 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f7a8406c8f0 0x7f7a8406edb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:05.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.983+0000 7f7aaad52640 1 --2- 172.21.15.44:0/49292861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7aa41552e0 0x7f7aa414ca20 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:05.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.983+0000 7f7aaad52640 1 --2- 172.21.15.44:0/49292861 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7aa4154f00 0x7f7aa414c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:05.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.983+0000 7f7aaad52640 1 --2- 172.21.15.44:0/49292861 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7aa406d470 0x7f7aa414a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:05.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.983+0000 7f7aaad52640 1 -- 172.21.15.44:0/49292861 >> 172.21.15.44:0/49292861 conn(0x7f7aa40733c0 msgr2=0x7f7aa40737a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:05.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.983+0000 7f7aaad52640 1 -- 172.21.15.44:0/49292861 shutdown_connections 2024-04-22T06:31:05.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:05.983+0000 7f7aaad52640 1 -- 172.21.15.44:0/49292861 wait complete. 2024-04-22T06:31:06.008 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-22T06:31:06.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.118+0000 7efff6662640 1 Processor -- start 2024-04-22T06:31:06.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.118+0000 7efff6662640 1 -- start start 2024-04-22T06:31:06.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.119+0000 7efff6662640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efff0141fc0 0x7efff00c5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:06.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.119+0000 7efff6662640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7efff00cd190 0x7efff00c6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:06.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.119+0000 7efff6662640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efff0159010 con 0x7efff01423a0 2024-04-22T06:31:06.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.119+0000 7efff6662640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7efff01592b0 con 0x7efff00cd190 2024-04-22T06:31:06.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.119+0000 7efff6662640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7efff0159590 con 0x7efff0141fc0 2024-04-22T06:31:06.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.119+0000 7effef7fe640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7efff01423a0 0x7efff00cca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:56358/0 (socket says 172.21.15.44:56358) 2024-04-22T06:31:06.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.119+0000 7effef7fe640 1 -- 172.21.15.44:0/1234615742 learned_addr learned my addr 172.21.15.44:0/1234615742 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:06.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.120+0000 7effeffff640 1 --2- 172.21.15.44:0/1234615742 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efff0141fc0 0x7efff00c5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:06.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.120+0000 7effeeffd640 1 -- 172.21.15.44:0/1234615742 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2692872415 0 0) 0x7efff0159010 con 0x7efff01423a0 2024-04-22T06:31:06.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.120+0000 7effeeffd640 1 -- 172.21.15.44:0/1234615742 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7effdc003520 con 0x7efff01423a0 2024-04-22T06:31:06.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.120+0000 7efff4bff640 1 --2- 172.21.15.44:0/1234615742 >> v2:172.21.15.70:3300/0 conn(0x7efff00cd190 0x7efff00c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:06.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.120+0000 7effeeffd640 1 -- 172.21.15.44:0/1234615742 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 765195262 0 0) 0x7effdc003520 con 0x7efff01423a0 2024-04-22T06:31:06.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.120+0000 7effeeffd640 1 -- 172.21.15.44:0/1234615742 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efff0141fc0 msgr2=0x7efff00c5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:06.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.120+0000 7effeeffd640 1 --2- 172.21.15.44:0/1234615742 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efff0141fc0 0x7efff00c5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:06.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.120+0000 7effeeffd640 1 -- 172.21.15.44:0/1234615742 >> v2:172.21.15.70:3300/0 conn(0x7efff00cd190 msgr2=0x7efff00c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:06.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.120+0000 7effeeffd640 1 --2- 172.21.15.44:0/1234615742 >> v2:172.21.15.70:3300/0 conn(0x7efff00cd190 0x7efff00c6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:06.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.120+0000 7effeeffd640 1 -- 172.21.15.44:0/1234615742 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efff0159870 con 0x7efff01423a0 2024-04-22T06:31:06.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.120+0000 7effeeffd640 1 -- 172.21.15.44:0/1234615742 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1237339257 0 0) 0x7effd8001670 con 0x7efff01423a0 2024-04-22T06:31:06.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.120+0000 7effeeffd640 1 -- 172.21.15.44:0/1234615742 >> v1:172.21.15.44:6789/0 conn(0x7efff01423a0 legacy=0x7efff00cca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:06.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.120+0000 7effeeffd640 1 --2- 172.21.15.44:0/1234615742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effdc0039b0 0x7effdc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:06.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.120+0000 7effeeffd640 1 --2- 172.21.15.44:0/1234615742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7effdc004800 0x7effdc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:06.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.120+0000 7effeeffd640 1 --2- 172.21.15.44:0/1234615742 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efff0141fc0 0x7effdc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:06.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.120+0000 7effeeffd640 1 -- 172.21.15.44:0/1234615742 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efff01592b0 con 0x7effdc0039b0 2024-04-22T06:31:06.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.120+0000 7effeeffd640 1 -- 172.21.15.44:0/1234615742 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7efff0159590 con 0x7effdc004800 2024-04-22T06:31:06.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.120+0000 7effeeffd640 1 -- 172.21.15.44:0/1234615742 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7effdc00b9c0 con 0x7efff0141fc0 2024-04-22T06:31:06.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.120+0000 7effeffff640 1 --2- 172.21.15.44:0/1234615742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effdc0039b0 0x7effdc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:06.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.120+0000 7effef7fe640 1 --2- 172.21.15.44:0/1234615742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7effdc004800 0x7effdc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:06.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.120+0000 7efff4bff640 1 --2- 172.21.15.44:0/1234615742 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efff0141fc0 0x7effdc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:06.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.120+0000 7effeffff640 1 -- 172.21.15.44:0/1234615742 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efff0141fc0 msgr2=0x7effdc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:06.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.120+0000 7effeffff640 1 --2- 172.21.15.44:0/1234615742 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efff0141fc0 0x7effdc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:06.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.120+0000 7effeffff640 1 -- 172.21.15.44:0/1234615742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7effdc004800 msgr2=0x7effdc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:06.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.120+0000 7effeffff640 1 --2- 172.21.15.44:0/1234615742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7effdc004800 0x7effdc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:06.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.120+0000 7effeffff640 1 -- 172.21.15.44:0/1234615742 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effdc003520 con 0x7effdc0039b0 2024-04-22T06:31:06.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.120+0000 7effef7fe640 1 --2- 172.21.15.44:0/1234615742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7effdc004800 0x7effdc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:31:06.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.121+0000 7effeffff640 1 --2- 172.21.15.44:0/1234615742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effdc0039b0 0x7effdc003dd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7effe0009ac0 tx=0x7effdc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f94a2cb78456a563 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:06.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.121+0000 7effeeffd640 1 -- 172.21.15.44:0/1234615742 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe0039070 con 0x7effdc0039b0 2024-04-22T06:31:06.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.121+0000 7effeeffd640 1 -- 172.21.15.44:0/1234615742 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effe002c6b0 con 0x7effdc0039b0 2024-04-22T06:31:06.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.121+0000 7effeeffd640 1 -- 172.21.15.44:0/1234615742 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe0009690 con 0x7effdc0039b0 2024-04-22T06:31:06.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.122+0000 7efff6662640 1 -- 172.21.15.44:0/1234615742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effdc0039b0 msgr2=0x7effdc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:06.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.122+0000 7efff6662640 1 --2- 172.21.15.44:0/1234615742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effdc0039b0 0x7effdc003dd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7effe0009ac0 tx=0x7effdc005830 comp rx=0 tx=0).stop 2024-04-22T06:31:06.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.122+0000 7efff4bff640 1 -- 172.21.15.44:0/1234615742 reap_dead start 2024-04-22T06:31:06.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.122+0000 7efff6662640 1 -- 172.21.15.44:0/1234615742 shutdown_connections 2024-04-22T06:31:06.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.122+0000 7efff6662640 1 -- 172.21.15.44:0/1234615742 >> 172.21.15.44:0/1234615742 conn(0x7efff006d530 msgr2=0x7efff0074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:06.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.122+0000 7efff6662640 1 -- 172.21.15.44:0/1234615742 shutdown_connections 2024-04-22T06:31:06.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.122+0000 7efff6662640 1 -- 172.21.15.44:0/1234615742 wait complete. 2024-04-22T06:31:06.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.123+0000 7efff6662640 1 Processor -- start 2024-04-22T06:31:06.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.123+0000 7efff6662640 1 -- start start 2024-04-22T06:31:06.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.124+0000 7efff6662640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7effdc0039b0 0x7efff013ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:06.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.124+0000 7efff6662640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efff00c46d0 0x7efff013c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:06.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.124+0000 7efff6662640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efff01f1670 0x7efff013d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:06.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.124+0000 7efff6662640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efff0071ca0 con 0x7efff00c46d0 2024-04-22T06:31:06.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.124+0000 7efff6662640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7efff0071b20 con 0x7efff01f1670 2024-04-22T06:31:06.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.124+0000 7efff6662640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7efff0071e20 con 0x7effdc0039b0 2024-04-22T06:31:06.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.124+0000 7effef7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efff00c46d0 0x7efff013c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:06.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.124+0000 7effef7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efff00c46d0 0x7efff013c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60612/0 (socket says 172.21.15.44:60612) 2024-04-22T06:31:06.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.124+0000 7effef7fe640 1 -- 172.21.15.44:0/4267776051 learned_addr learned my addr 172.21.15.44:0/4267776051 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:06.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.124+0000 7effeffff640 1 --2- 172.21.15.44:0/4267776051 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7effdc0039b0 0x7efff013ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:06.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.124+0000 7efff4bff640 1 --2- 172.21.15.44:0/4267776051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efff01f1670 0x7efff013d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:06.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.124+0000 7effef7fe640 1 -- 172.21.15.44:0/4267776051 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7effdc0039b0 msgr2=0x7efff013ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:06.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.124+0000 7effef7fe640 1 --2- 172.21.15.44:0/4267776051 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7effdc0039b0 0x7efff013ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:06.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.124+0000 7effef7fe640 1 -- 172.21.15.44:0/4267776051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efff01f1670 msgr2=0x7efff013d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:06.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.124+0000 7effef7fe640 1 --2- 172.21.15.44:0/4267776051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efff01f1670 0x7efff013d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:06.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.124+0000 7effef7fe640 1 -- 172.21.15.44:0/4267776051 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efff013bbc0 con 0x7efff00c46d0 2024-04-22T06:31:06.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.125+0000 7effef7fe640 1 --2- 172.21.15.44:0/4267776051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efff00c46d0 0x7efff013c6a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7effd8000ff0 tx=0x7effd800eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:06.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.125+0000 7effecff9640 1 -- 172.21.15.44:0/4267776051 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd800c5c0 con 0x7efff00c46d0 2024-04-22T06:31:06.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.125+0000 7effecff9640 1 -- 172.21.15.44:0/4267776051 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effd8018070 con 0x7efff00c46d0 2024-04-22T06:31:06.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.125+0000 7efff6662640 1 -- 172.21.15.44:0/4267776051 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efff013d910 con 0x7efff00c46d0 2024-04-22T06:31:06.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.125+0000 7effecff9640 1 -- 172.21.15.44:0/4267776051 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd800c960 con 0x7efff00c46d0 2024-04-22T06:31:06.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.125+0000 7efff6662640 1 -- 172.21.15.44:0/4267776051 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efff00c3910 con 0x7efff00c46d0 2024-04-22T06:31:06.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.126+0000 7efff6662640 1 -- 172.21.15.44:0/4267776051 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efff0068cc0 con 0x7efff00c46d0 2024-04-22T06:31:06.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.126+0000 7effecff9640 1 -- 172.21.15.44:0/4267776051 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7effd800cb00 con 0x7efff00c46d0 2024-04-22T06:31:06.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.127+0000 7effecff9640 1 --2- 172.21.15.44:0/4267776051 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7effd006c8f0 0x7effd006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:06.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.127+0000 7effecff9640 1 -- 172.21.15.44:0/4267776051 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7effd808b680 con 0x7efff00c46d0 2024-04-22T06:31:06.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.127+0000 7effeffff640 1 --2- 172.21.15.44:0/4267776051 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7effd006c8f0 0x7effd006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:06.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.127+0000 7effeffff640 1 --2- 172.21.15.44:0/4267776051 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7effd006c8f0 0x7effd006edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7effe0036000 tx=0x7effe0030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:06.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.132+0000 7effecff9640 1 -- 172.21.15.44:0/4267776051 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7effd8059550 con 0x7efff00c46d0 2024-04-22T06:31:06.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:06.355+0000 7efff6662640 1 -- 172.21.15.44:0/4267776051 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7efff00c35a0 con 0x7efff00c46d0 2024-04-22T06:31:07.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.015+0000 7effecff9640 1 -- 172.21.15.44:0/4267776051 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7effd805d850 con 0x7efff00c46d0 2024-04-22T06:31:07.014 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 2 and data pool 3 2024-04-22T06:31:07.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.023+0000 7efff6662640 1 -- 172.21.15.44:0/4267776051 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7effd006c8f0 msgr2=0x7effd006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:07.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.023+0000 7efff6662640 1 --2- 172.21.15.44:0/4267776051 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7effd006c8f0 0x7effd006edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7effe0036000 tx=0x7effe0030000 comp rx=0 tx=0).stop 2024-04-22T06:31:07.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.023+0000 7efff6662640 1 -- 172.21.15.44:0/4267776051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efff00c46d0 msgr2=0x7efff013c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:07.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.023+0000 7efff6662640 1 --2- 172.21.15.44:0/4267776051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efff00c46d0 0x7efff013c6a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7effd8000ff0 tx=0x7effd800eae0 comp rx=0 tx=0).stop 2024-04-22T06:31:07.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.024+0000 7efff6662640 1 -- 172.21.15.44:0/4267776051 shutdown_connections 2024-04-22T06:31:07.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.024+0000 7efff6662640 1 --2- 172.21.15.44:0/4267776051 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7effd006c8f0 0x7effd006edb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:07.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.024+0000 7efff6662640 1 --2- 172.21.15.44:0/4267776051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efff01f1670 0x7efff013d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:07.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.024+0000 7efff6662640 1 --2- 172.21.15.44:0/4267776051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efff00c46d0 0x7efff013c6a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:07.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.024+0000 7efff6662640 1 --2- 172.21.15.44:0/4267776051 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7effdc0039b0 0x7efff013ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:07.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.024+0000 7efff6662640 1 -- 172.21.15.44:0/4267776051 >> 172.21.15.44:0/4267776051 conn(0x7efff006d530 msgr2=0x7efff01411e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:07.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.025+0000 7efff6662640 1 -- 172.21.15.44:0/4267776051 shutdown_connections 2024-04-22T06:31:07.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.025+0000 7efff6662640 1 -- 172.21.15.44:0/4267776051 wait complete. 2024-04-22T06:31:07.039 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-22T06:31:07.039 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-22T06:31:07.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.151+0000 7fb72d069640 1 Processor -- start 2024-04-22T06:31:07.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.151+0000 7fb72d069640 1 -- start start 2024-04-22T06:31:07.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.152+0000 7fb72d069640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb728153000 0x7fb7280ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:07.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.152+0000 7fb72d069640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7280cd0b0 0x7fb7280cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:07.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.152+0000 7fb72d069640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7280c4c80 con 0x7fb7280cc4a0 2024-04-22T06:31:07.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.152+0000 7fb72d069640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fb7280ce610 con 0x7fb728153000 2024-04-22T06:31:07.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.152+0000 7fb72d069640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb7280c5570 con 0x7fb7280cd0b0 2024-04-22T06:31:07.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.152+0000 7fb726d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb728153000 0x7fb7280ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:07.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.152+0000 7fb71ffff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7280cd0b0 0x7fb7280cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:07.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.152+0000 7fb726d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb728153000 0x7fb7280ce0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:57484/0 (socket says 172.21.15.44:57484) 2024-04-22T06:31:07.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.152+0000 7fb726d9d640 1 -- 172.21.15.44:0/632755441 learned_addr learned my addr 172.21.15.44:0/632755441 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:07.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.152+0000 7fb72759e640 1 --1- 172.21.15.44:0/632755441 >> v1:172.21.15.70:6789/0 conn(0x7fb7280cc4a0 0x7fb7280c6e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.44:54348/0 (socket says 172.21.15.44:54348) 2024-04-22T06:31:07.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.153+0000 7fb71ffff640 1 -- 172.21.15.44:0/632755441 >> v2:172.21.15.70:3300/0 conn(0x7fb728153000 msgr2=0x7fb7280ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:07.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.153+0000 7fb71ffff640 1 --2- 172.21.15.44:0/632755441 >> v2:172.21.15.70:3300/0 conn(0x7fb728153000 0x7fb7280ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:07.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.153+0000 7fb71ffff640 1 -- 172.21.15.44:0/632755441 >> v1:172.21.15.70:6789/0 conn(0x7fb7280cc4a0 legacy=0x7fb7280c6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:07.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.153+0000 7fb71ffff640 1 -- 172.21.15.44:0/632755441 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7280cddf0 con 0x7fb7280cd0b0 2024-04-22T06:31:07.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.153+0000 7fb726d9d640 1 --2- 172.21.15.44:0/632755441 >> v2:172.21.15.70:3300/0 conn(0x7fb728153000 0x7fb7280ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:31:07.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.154+0000 7fb71ffff640 1 --2- 172.21.15.44:0/632755441 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7280cd0b0 0x7fb7280cbf60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb7100097a0 tx=0x7fb71002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d408157f6d6d1b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:07.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.154+0000 7fb72659c640 1 -- 172.21.15.44:0/632755441 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb710035020 con 0x7fb7280cd0b0 2024-04-22T06:31:07.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.154+0000 7fb72659c640 1 -- 172.21.15.44:0/632755441 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb710031830 con 0x7fb7280cd0b0 2024-04-22T06:31:07.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.154+0000 7fb72659c640 1 -- 172.21.15.44:0/632755441 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb710031c00 con 0x7fb7280cd0b0 2024-04-22T06:31:07.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.154+0000 7fb72d069640 1 -- 172.21.15.44:0/632755441 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7280cd0b0 msgr2=0x7fb7280cbf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:07.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.154+0000 7fb72d069640 1 --2- 172.21.15.44:0/632755441 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7280cd0b0 0x7fb7280cbf60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb7100097a0 tx=0x7fb71002b8d0 comp rx=0 tx=0).stop 2024-04-22T06:31:07.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.154+0000 7fb72d069640 1 -- 172.21.15.44:0/632755441 shutdown_connections 2024-04-22T06:31:07.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.154+0000 7fb72d069640 1 --2- 172.21.15.44:0/632755441 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7280cd0b0 0x7fb7280cbf60 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:07.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.154+0000 7fb72d069640 1 --2- 172.21.15.44:0/632755441 >> v2:172.21.15.70:3300/0 conn(0x7fb728153000 0x7fb7280ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:07.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.154+0000 7fb72d069640 1 -- 172.21.15.44:0/632755441 >> 172.21.15.44:0/632755441 conn(0x7fb728068370 msgr2=0x7fb72806fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:07.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.154+0000 7fb72d069640 1 -- 172.21.15.44:0/632755441 shutdown_connections 2024-04-22T06:31:07.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.154+0000 7fb72d069640 1 -- 172.21.15.44:0/632755441 wait complete. 2024-04-22T06:31:07.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.156+0000 7fb72d069640 1 Processor -- start 2024-04-22T06:31:07.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.156+0000 7fb72d069640 1 -- start start 2024-04-22T06:31:07.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.156+0000 7fb72d069640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7280cd0b0 0x7fb7280c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:07.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.156+0000 7fb72d069640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb728153000 0x7fb7281f15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:07.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.156+0000 7fb72d069640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb728159c50 0x7fb728073790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:07.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.156+0000 7fb72d069640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb7280cda90 con 0x7fb7280cd0b0 2024-04-22T06:31:07.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.156+0000 7fb72d069640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb7280c5200 con 0x7fb728153000 2024-04-22T06:31:07.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.156+0000 7fb72d069640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb7281533e0 con 0x7fb728159c50 2024-04-22T06:31:07.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.156+0000 7fb726d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7280cd0b0 0x7fb7280c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:07.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.156+0000 7fb726d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7280cd0b0 0x7fb7280c8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60616/0 (socket says 172.21.15.44:60616) 2024-04-22T06:31:07.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.156+0000 7fb726d9d640 1 -- 172.21.15.44:0/668752427 learned_addr learned my addr 172.21.15.44:0/668752427 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:07.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.156+0000 7fb71ffff640 1 --2- 172.21.15.44:0/668752427 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb728153000 0x7fb7281f15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:07.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.156+0000 7fb72759e640 1 --2- 172.21.15.44:0/668752427 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb728159c50 0x7fb728073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:07.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.157+0000 7fb726d9d640 1 -- 172.21.15.44:0/668752427 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb728159c50 msgr2=0x7fb728073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:07.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.157+0000 7fb726d9d640 1 --2- 172.21.15.44:0/668752427 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb728159c50 0x7fb728073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:07.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.157+0000 7fb726d9d640 1 -- 172.21.15.44:0/668752427 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb728153000 msgr2=0x7fb7281f15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:07.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.157+0000 7fb726d9d640 1 --2- 172.21.15.44:0/668752427 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb728153000 0x7fb7281f15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:07.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.157+0000 7fb726d9d640 1 -- 172.21.15.44:0/668752427 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7281f0b10 con 0x7fb7280cd0b0 2024-04-22T06:31:07.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.157+0000 7fb726d9d640 1 --2- 172.21.15.44:0/668752427 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7280cd0b0 0x7fb7280c8bf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb71800ea10 tx=0x7fb71800eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:07.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.157+0000 7fb71f7fe640 1 -- 172.21.15.44:0/668752427 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb718017070 con 0x7fb7280cd0b0 2024-04-22T06:31:07.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.157+0000 7fb71f7fe640 1 -- 172.21.15.44:0/668752427 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb71800ca00 con 0x7fb7280cd0b0 2024-04-22T06:31:07.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.157+0000 7fb72d069640 1 -- 172.21.15.44:0/668752427 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb728072e70 con 0x7fb7280cd0b0 2024-04-22T06:31:07.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.157+0000 7fb71f7fe640 1 -- 172.21.15.44:0/668752427 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71800cdb0 con 0x7fb7280cd0b0 2024-04-22T06:31:07.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.157+0000 7fb72d069640 1 -- 172.21.15.44:0/668752427 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7281f0dd0 con 0x7fb7280cd0b0 2024-04-22T06:31:07.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.158+0000 7fb72d069640 1 -- 172.21.15.44:0/668752427 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb728069310 con 0x7fb7280cd0b0 2024-04-22T06:31:07.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.159+0000 7fb71f7fe640 1 -- 172.21.15.44:0/668752427 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb71801a070 con 0x7fb7280cd0b0 2024-04-22T06:31:07.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.159+0000 7fb71f7fe640 1 --2- 172.21.15.44:0/668752427 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb70806c920 0x7fb70806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:07.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.159+0000 7fb71f7fe640 1 -- 172.21.15.44:0/668752427 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb71808b880 con 0x7fb7280cd0b0 2024-04-22T06:31:07.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.159+0000 7fb71ffff640 1 --2- 172.21.15.44:0/668752427 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb70806c920 0x7fb70806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:07.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.160+0000 7fb71ffff640 1 --2- 172.21.15.44:0/668752427 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb70806c920 0x7fb70806ede0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb710036000 tx=0x7fb710008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:07.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.164+0000 7fb71f7fe640 1 -- 172.21.15.44:0/668752427 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb7180596f0 con 0x7fb7280cd0b0 2024-04-22T06:31:07.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:07.357+0000 7fb72d069640 1 -- 172.21.15.44:0/668752427 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fb7281515b0 con 0x7fb7280cd0b0 2024-04-22T06:31:08.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.007+0000 7fb71f7fe640 1 -- 172.21.15.44:0/668752427 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) ==== 157+0+0 (secure 0 0 0) 0x7fb71805d9f0 con 0x7fb7280cd0b0 2024-04-22T06:31:08.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.011+0000 7fb72d069640 1 -- 172.21.15.44:0/668752427 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb70806c920 msgr2=0x7fb70806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:08.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.011+0000 7fb72d069640 1 --2- 172.21.15.44:0/668752427 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb70806c920 0x7fb70806ede0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb710036000 tx=0x7fb710008030 comp rx=0 tx=0).stop 2024-04-22T06:31:08.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.011+0000 7fb72d069640 1 -- 172.21.15.44:0/668752427 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7280cd0b0 msgr2=0x7fb7280c8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:08.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.011+0000 7fb72d069640 1 --2- 172.21.15.44:0/668752427 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7280cd0b0 0x7fb7280c8bf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb71800ea10 tx=0x7fb71800eed0 comp rx=0 tx=0).stop 2024-04-22T06:31:08.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.011+0000 7fb72d069640 1 -- 172.21.15.44:0/668752427 shutdown_connections 2024-04-22T06:31:08.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.011+0000 7fb72d069640 1 --2- 172.21.15.44:0/668752427 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb70806c920 0x7fb70806ede0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:08.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.011+0000 7fb72d069640 1 --2- 172.21.15.44:0/668752427 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb728159c50 0x7fb728073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:08.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.011+0000 7fb72d069640 1 --2- 172.21.15.44:0/668752427 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb728153000 0x7fb7281f15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:08.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.011+0000 7fb72d069640 1 --2- 172.21.15.44:0/668752427 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7280cd0b0 0x7fb7280c8bf0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:08.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.011+0000 7fb72d069640 1 -- 172.21.15.44:0/668752427 >> 172.21.15.44:0/668752427 conn(0x7fb7280cc4a0 msgr2=0x7fb728152150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:08.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.011+0000 7fb72d069640 1 -- 172.21.15.44:0/668752427 shutdown_connections 2024-04-22T06:31:08.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.011+0000 7fb72d069640 1 -- 172.21.15.44:0/668752427 wait complete. 2024-04-22T06:31:08.022 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-22T06:31:08.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.135+0000 7ff459323640 1 Processor -- start 2024-04-22T06:31:08.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.135+0000 7ff459323640 1 -- start start 2024-04-22T06:31:08.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.136+0000 7ff459323640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4540cb580 0x7ff4540d0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:08.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.136+0000 7ff459323640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff45415ec10 0x7ff45415f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:08.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.136+0000 7ff459323640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff45415bf90 con 0x7ff4540d0a00 2024-04-22T06:31:08.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.136+0000 7ff459323640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7ff45415e780 con 0x7ff45415ec10 2024-04-22T06:31:08.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.136+0000 7ff459323640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ff45415b410 con 0x7ff4540cb580 2024-04-22T06:31:08.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.136+0000 7ff452ffd640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7ff4540d0a00 0x7ff4540cccc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:56360/0 (socket says 172.21.15.44:56360) 2024-04-22T06:31:08.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.136+0000 7ff452ffd640 1 -- 172.21.15.44:0/1468300155 learned_addr learned my addr 172.21.15.44:0/1468300155 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:08.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.137+0000 7ff4537fe640 1 --2- 172.21.15.44:0/1468300155 >> v2:172.21.15.70:3300/0 conn(0x7ff45415ec10 0x7ff45415f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:08.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.137+0000 7ff4527fc640 1 --2- 172.21.15.44:0/1468300155 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4540cb580 0x7ff4540d0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:08.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.137+0000 7ff451ffb640 1 -- 172.21.15.44:0/1468300155 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4152871785 0 0) 0x7ff44c001110 con 0x7ff4540d0a00 2024-04-22T06:31:08.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.137+0000 7ff451ffb640 1 -- 172.21.15.44:0/1468300155 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff440003520 con 0x7ff4540d0a00 2024-04-22T06:31:08.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.137+0000 7ff451ffb640 1 -- 172.21.15.44:0/1468300155 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 890408823 0 0) 0x7ff440003520 con 0x7ff4540d0a00 2024-04-22T06:31:08.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.137+0000 7ff451ffb640 1 -- 172.21.15.44:0/1468300155 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4540cb580 msgr2=0x7ff4540d0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:08.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.137+0000 7ff451ffb640 1 --2- 172.21.15.44:0/1468300155 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4540cb580 0x7ff4540d0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:08.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.137+0000 7ff451ffb640 1 -- 172.21.15.44:0/1468300155 >> v2:172.21.15.70:3300/0 conn(0x7ff45415ec10 msgr2=0x7ff45415f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:08.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.137+0000 7ff451ffb640 1 --2- 172.21.15.44:0/1468300155 >> v2:172.21.15.70:3300/0 conn(0x7ff45415ec10 0x7ff45415f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:08.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.137+0000 7ff451ffb640 1 -- 172.21.15.44:0/1468300155 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff45415b6f0 con 0x7ff4540d0a00 2024-04-22T06:31:08.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.137+0000 7ff451ffb640 1 -- 172.21.15.44:0/1468300155 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1237339257 0 0) 0x7ff44c001840 con 0x7ff4540d0a00 2024-04-22T06:31:08.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.137+0000 7ff451ffb640 1 -- 172.21.15.44:0/1468300155 >> v1:172.21.15.44:6789/0 conn(0x7ff4540d0a00 legacy=0x7ff4540cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:08.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.137+0000 7ff451ffb640 1 --2- 172.21.15.44:0/1468300155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4400039b0 0x7ff440003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:08.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.137+0000 7ff451ffb640 1 --2- 172.21.15.44:0/1468300155 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4540cb580 0x7ff440004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:08.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.137+0000 7ff451ffb640 1 --2- 172.21.15.44:0/1468300155 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff440004d40 0x7ff4400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:08.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.137+0000 7ff451ffb640 1 -- 172.21.15.44:0/1468300155 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff45415e780 con 0x7ff4400039b0 2024-04-22T06:31:08.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.137+0000 7ff451ffb640 1 -- 172.21.15.44:0/1468300155 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff45415b410 con 0x7ff440004d40 2024-04-22T06:31:08.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.137+0000 7ff451ffb640 1 -- 172.21.15.44:0/1468300155 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ff44000b9c0 con 0x7ff4540cb580 2024-04-22T06:31:08.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.137+0000 7ff452ffd640 1 --2- 172.21.15.44:0/1468300155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4400039b0 0x7ff440003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:08.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.137+0000 7ff4527fc640 1 --2- 172.21.15.44:0/1468300155 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4540cb580 0x7ff440004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:08.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.137+0000 7ff4537fe640 1 --2- 172.21.15.44:0/1468300155 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff440004d40 0x7ff4400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:08.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.137+0000 7ff452ffd640 1 -- 172.21.15.44:0/1468300155 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4540cb580 msgr2=0x7ff440004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:08.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.137+0000 7ff452ffd640 1 --2- 172.21.15.44:0/1468300155 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4540cb580 0x7ff440004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:08.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.137+0000 7ff452ffd640 1 -- 172.21.15.44:0/1468300155 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff440004d40 msgr2=0x7ff4400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:08.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.137+0000 7ff452ffd640 1 --2- 172.21.15.44:0/1468300155 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff440004d40 0x7ff4400051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:08.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.137+0000 7ff452ffd640 1 -- 172.21.15.44:0/1468300155 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff440003520 con 0x7ff4400039b0 2024-04-22T06:31:08.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.137+0000 7ff4537fe640 1 --2- 172.21.15.44:0/1468300155 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff440004d40 0x7ff4400051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:31:08.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.138+0000 7ff452ffd640 1 --2- 172.21.15.44:0/1468300155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4400039b0 0x7ff440003dd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff44c00c3e0 tx=0x7ff440005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b55470c0a0311c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:08.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.138+0000 7ff451ffb640 1 -- 172.21.15.44:0/1468300155 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44c00c0e0 con 0x7ff4400039b0 2024-04-22T06:31:08.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.138+0000 7ff451ffb640 1 -- 172.21.15.44:0/1468300155 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff44c030070 con 0x7ff4400039b0 2024-04-22T06:31:08.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.138+0000 7ff451ffb640 1 -- 172.21.15.44:0/1468300155 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44c0058d0 con 0x7ff4400039b0 2024-04-22T06:31:08.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.139+0000 7ff459323640 1 -- 172.21.15.44:0/1468300155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4400039b0 msgr2=0x7ff440003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:08.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.139+0000 7ff459323640 1 --2- 172.21.15.44:0/1468300155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4400039b0 0x7ff440003dd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff44c00c3e0 tx=0x7ff440005830 comp rx=0 tx=0).stop 2024-04-22T06:31:08.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.139+0000 7ff4537fe640 1 -- 172.21.15.44:0/1468300155 reap_dead start 2024-04-22T06:31:08.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.139+0000 7ff459323640 1 -- 172.21.15.44:0/1468300155 shutdown_connections 2024-04-22T06:31:08.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.139+0000 7ff459323640 1 -- 172.21.15.44:0/1468300155 >> 172.21.15.44:0/1468300155 conn(0x7ff454072340 msgr2=0x7ff454074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:08.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.139+0000 7ff459323640 1 -- 172.21.15.44:0/1468300155 shutdown_connections 2024-04-22T06:31:08.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.139+0000 7ff459323640 1 -- 172.21.15.44:0/1468300155 wait complete. 2024-04-22T06:31:08.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.140+0000 7ff459323640 1 Processor -- start 2024-04-22T06:31:08.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.140+0000 7ff459323640 1 -- start start 2024-04-22T06:31:08.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.141+0000 7ff459323640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4400039b0 0x7ff4541f1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:08.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.141+0000 7ff459323640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff45415dec0 0x7ff45414f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:08.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.141+0000 7ff459323640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff45415c6e0 0x7ff4540b6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:08.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.141+0000 7ff459323640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff454155610 con 0x7ff45415dec0 2024-04-22T06:31:08.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.141+0000 7ff459323640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff45415bcb0 con 0x7ff45415c6e0 2024-04-22T06:31:08.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.141+0000 7ff459323640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ff454155790 con 0x7ff4400039b0 2024-04-22T06:31:08.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.141+0000 7ff4527fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff45415dec0 0x7ff45414f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:08.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.141+0000 7ff4527fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff45415dec0 0x7ff45414f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60626/0 (socket says 172.21.15.44:60626) 2024-04-22T06:31:08.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.141+0000 7ff4527fc640 1 -- 172.21.15.44:0/2403716496 learned_addr learned my addr 172.21.15.44:0/2403716496 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:08.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.141+0000 7ff4537fe640 1 --2- 172.21.15.44:0/2403716496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff45415c6e0 0x7ff4540b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:08.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.141+0000 7ff452ffd640 1 --2- 172.21.15.44:0/2403716496 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4400039b0 0x7ff4541f1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:08.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.141+0000 7ff4527fc640 1 -- 172.21.15.44:0/2403716496 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4400039b0 msgr2=0x7ff4541f1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:08.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.141+0000 7ff4527fc640 1 --2- 172.21.15.44:0/2403716496 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4400039b0 0x7ff4541f1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:08.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.141+0000 7ff4527fc640 1 -- 172.21.15.44:0/2403716496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff45415c6e0 msgr2=0x7ff4540b6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:08.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.141+0000 7ff4527fc640 1 --2- 172.21.15.44:0/2403716496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff45415c6e0 0x7ff4540b6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:08.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.141+0000 7ff4527fc640 1 -- 172.21.15.44:0/2403716496 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4541f0ee0 con 0x7ff45415dec0 2024-04-22T06:31:08.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.141+0000 7ff4537fe640 1 --2- 172.21.15.44:0/2403716496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff45415c6e0 0x7ff4540b6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:31:08.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.141+0000 7ff452ffd640 1 --2- 172.21.15.44:0/2403716496 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4400039b0 0x7ff4541f1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:31:08.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.141+0000 7ff4527fc640 1 --2- 172.21.15.44:0/2403716496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff45415dec0 0x7ff45414f8b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff43c011a00 tx=0x7ff43c011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:08.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.141+0000 7ff44b7fe640 1 -- 172.21.15.44:0/2403716496 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43c00fb50 con 0x7ff45415dec0 2024-04-22T06:31:08.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.142+0000 7ff44b7fe640 1 -- 172.21.15.44:0/2403716496 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff43c00fcf0 con 0x7ff45415dec0 2024-04-22T06:31:08.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.142+0000 7ff44b7fe640 1 -- 172.21.15.44:0/2403716496 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43c00fea0 con 0x7ff45415dec0 2024-04-22T06:31:08.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.142+0000 7ff459323640 1 -- 172.21.15.44:0/2403716496 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4541f4ef0 con 0x7ff45415dec0 2024-04-22T06:31:08.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.142+0000 7ff459323640 1 -- 172.21.15.44:0/2403716496 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4541f51d0 con 0x7ff45415dec0 2024-04-22T06:31:08.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.143+0000 7ff459323640 1 -- 172.21.15.44:0/2403716496 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff454076220 con 0x7ff45415dec0 2024-04-22T06:31:08.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.143+0000 7ff44b7fe640 1 -- 172.21.15.44:0/2403716496 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff43c012040 con 0x7ff45415dec0 2024-04-22T06:31:08.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.143+0000 7ff44b7fe640 1 --2- 172.21.15.44:0/2403716496 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7ff43406c8f0 0x7ff43406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:08.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.144+0000 7ff44b7fe640 1 -- 172.21.15.44:0/2403716496 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 6007+0+0 (secure 0 0 0) 0x7ff43c08c3d0 con 0x7ff45415dec0 2024-04-22T06:31:08.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.144+0000 7ff452ffd640 1 --2- 172.21.15.44:0/2403716496 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7ff43406c8f0 0x7ff43406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:08.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.144+0000 7ff452ffd640 1 --2- 172.21.15.44:0/2403716496 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7ff43406c8f0 0x7ff43406edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff44c00c3b0 tx=0x7ff44c007000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:08.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.148+0000 7ff44b7fe640 1 -- 172.21.15.44:0/2403716496 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff43c019030 con 0x7ff45415dec0 2024-04-22T06:31:08.326 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T06:31:08.329 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T06:31:08.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:08.334+0000 7ff459323640 1 -- 172.21.15.44:0/2403716496 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ff45414b810 con 0x7ff45415dec0 2024-04-22T06:31:10.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.020+0000 7ff44b7fe640 1 -- 172.21.15.44:0/2403716496 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) ==== 228+0+0 (secure 0 0 0) 0x7ff43c05a6d0 con 0x7ff45415dec0 2024-04-22T06:31:10.019 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data_ec' created 2024-04-22T06:31:10.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.023+0000 7ff459323640 1 -- 172.21.15.44:0/2403716496 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7ff43406c8f0 msgr2=0x7ff43406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:10.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.023+0000 7ff459323640 1 --2- 172.21.15.44:0/2403716496 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7ff43406c8f0 0x7ff43406edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff44c00c3b0 tx=0x7ff44c007000 comp rx=0 tx=0).stop 2024-04-22T06:31:10.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.023+0000 7ff459323640 1 -- 172.21.15.44:0/2403716496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff45415dec0 msgr2=0x7ff45414f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:10.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.023+0000 7ff459323640 1 --2- 172.21.15.44:0/2403716496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff45415dec0 0x7ff45414f8b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff43c011a00 tx=0x7ff43c011ec0 comp rx=0 tx=0).stop 2024-04-22T06:31:10.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.024+0000 7ff459323640 1 -- 172.21.15.44:0/2403716496 shutdown_connections 2024-04-22T06:31:10.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.024+0000 7ff459323640 1 --2- 172.21.15.44:0/2403716496 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7ff43406c8f0 0x7ff43406edb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:10.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.024+0000 7ff459323640 1 --2- 172.21.15.44:0/2403716496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff45415c6e0 0x7ff4540b6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:10.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.024+0000 7ff459323640 1 --2- 172.21.15.44:0/2403716496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff45415dec0 0x7ff45414f8b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:10.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.024+0000 7ff459323640 1 --2- 172.21.15.44:0/2403716496 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4400039b0 0x7ff4541f1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:10.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.024+0000 7ff459323640 1 -- 172.21.15.44:0/2403716496 >> 172.21.15.44:0/2403716496 conn(0x7ff454072340 msgr2=0x7ff45414a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:10.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.024+0000 7ff459323640 1 -- 172.21.15.44:0/2403716496 shutdown_connections 2024-04-22T06:31:10.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.024+0000 7ff459323640 1 -- 172.21.15.44:0/2403716496 wait complete. 2024-04-22T06:31:10.043 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-22T06:31:10.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.155+0000 7f1e0034a640 1 Processor -- start 2024-04-22T06:31:10.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.155+0000 7f1e0034a640 1 -- start start 2024-04-22T06:31:10.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.156+0000 7f1e0034a640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1df80d0e80 0x7f1df806d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:10.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.156+0000 7f1e0034a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1df80d1260 0x7f1df8154e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:10.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.156+0000 7f1e0034a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1df815f330 con 0x7f1df80cc8b0 2024-04-22T06:31:10.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.156+0000 7f1e0034a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f1df80c5ba0 con 0x7f1df80d1260 2024-04-22T06:31:10.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.156+0000 7f1e0034a640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f1df80cb410 con 0x7f1df80d0e80 2024-04-22T06:31:10.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.156+0000 7f1dfe0e6640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1df80d0e80 0x7f1df806d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:10.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.156+0000 7f1dfd8e5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1df80d1260 0x7f1df8154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:10.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.156+0000 7f1dfe0e6640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1df80d0e80 0x7f1df806d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.44:47272/0 (socket says 172.21.15.44:47272) 2024-04-22T06:31:10.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.156+0000 7f1dfd8e5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1df80d1260 0x7f1df8154e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:57518/0 (socket says 172.21.15.44:57518) 2024-04-22T06:31:10.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.156+0000 7f1dfe0e6640 1 -- 172.21.15.44:0/3011779953 learned_addr learned my addr 172.21.15.44:0/3011779953 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:10.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.156+0000 7f1dfe8e7640 1 --1- 172.21.15.44:0/3011779953 >> v1:172.21.15.70:6789/0 conn(0x7f1df80cc8b0 0x7f1df80c6030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.44:54352/0 (socket says 172.21.15.44:54352) 2024-04-22T06:31:10.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.157+0000 7f1dfe0e6640 1 -- 172.21.15.44:0/3011779953 >> v2:172.21.15.70:3300/0 conn(0x7f1df80d1260 msgr2=0x7f1df8154e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:10.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.157+0000 7f1dfe0e6640 1 --2- 172.21.15.44:0/3011779953 >> v2:172.21.15.70:3300/0 conn(0x7f1df80d1260 0x7f1df8154e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:10.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.157+0000 7f1dfe0e6640 1 -- 172.21.15.44:0/3011779953 >> v1:172.21.15.70:6789/0 conn(0x7f1df80cc8b0 legacy=0x7f1df80c6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:10.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.157+0000 7f1dfe0e6640 1 -- 172.21.15.44:0/3011779953 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df80cb6f0 con 0x7f1df80d0e80 2024-04-22T06:31:10.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.157+0000 7f1dfd8e5640 1 --2- 172.21.15.44:0/3011779953 >> v2:172.21.15.70:3300/0 conn(0x7f1df80d1260 0x7f1df8154e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:31:10.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.157+0000 7f1dfe0e6640 1 --2- 172.21.15.44:0/3011779953 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1df80d0e80 0x7f1df806d4f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1df4009870 tx=0x7f1df402ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8474d2381159bf99 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:10.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.157+0000 7f1dfd0e4640 1 -- 172.21.15.44:0/3011779953 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df402f830 con 0x7f1df80d0e80 2024-04-22T06:31:10.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.158+0000 7f1dfd0e4640 1 -- 172.21.15.44:0/3011779953 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1df402f9d0 con 0x7f1df80d0e80 2024-04-22T06:31:10.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.158+0000 7f1dfd0e4640 1 -- 172.21.15.44:0/3011779953 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df402fda0 con 0x7f1df80d0e80 2024-04-22T06:31:10.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.158+0000 7f1e0034a640 1 -- 172.21.15.44:0/3011779953 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1df80d0e80 msgr2=0x7f1df806d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:10.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.158+0000 7f1e0034a640 1 --2- 172.21.15.44:0/3011779953 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1df80d0e80 0x7f1df806d4f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1df4009870 tx=0x7f1df402ba10 comp rx=0 tx=0).stop 2024-04-22T06:31:10.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.158+0000 7f1e0034a640 1 -- 172.21.15.44:0/3011779953 shutdown_connections 2024-04-22T06:31:10.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.158+0000 7f1e0034a640 1 --2- 172.21.15.44:0/3011779953 >> v2:172.21.15.70:3300/0 conn(0x7f1df80d1260 0x7f1df8154e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:10.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.158+0000 7f1e0034a640 1 --2- 172.21.15.44:0/3011779953 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1df80d0e80 0x7f1df806d4f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:10.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.158+0000 7f1e0034a640 1 -- 172.21.15.44:0/3011779953 >> 172.21.15.44:0/3011779953 conn(0x7f1df8074e90 msgr2=0x7f1df80752a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:10.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.158+0000 7f1e0034a640 1 -- 172.21.15.44:0/3011779953 shutdown_connections 2024-04-22T06:31:10.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.158+0000 7f1e0034a640 1 -- 172.21.15.44:0/3011779953 wait complete. 2024-04-22T06:31:10.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.159+0000 7f1e0034a640 1 Processor -- start 2024-04-22T06:31:10.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.159+0000 7f1e0034a640 1 -- start start 2024-04-22T06:31:10.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.160+0000 7f1e0034a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1df80cc8b0 0x7f1df8153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:10.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.160+0000 7f1e0034a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1df80d0e80 0x7f1df80c9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:10.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.160+0000 7f1e0034a640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1df80d1260 0x7f1df80c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:10.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.160+0000 7f1e0034a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1df815efc0 con 0x7f1df80cc8b0 2024-04-22T06:31:10.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.160+0000 7f1e0034a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f1df80b3850 con 0x7f1df80d0e80 2024-04-22T06:31:10.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.160+0000 7f1e0034a640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f1df80c6740 con 0x7f1df80d1260 2024-04-22T06:31:10.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.160+0000 7f1dfe0e6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1df80cc8b0 0x7f1df8153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:10.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.160+0000 7f1dfe0e6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1df80cc8b0 0x7f1df8153d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60628/0 (socket says 172.21.15.44:60628) 2024-04-22T06:31:10.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.160+0000 7f1dfe0e6640 1 -- 172.21.15.44:0/3077402477 learned_addr learned my addr 172.21.15.44:0/3077402477 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:10.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.160+0000 7f1dfd8e5640 1 --2- 172.21.15.44:0/3077402477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1df80d0e80 0x7f1df80c9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:10.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.160+0000 7f1dfe8e7640 1 --2- 172.21.15.44:0/3077402477 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1df80d1260 0x7f1df80c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:10.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.161+0000 7f1dfe0e6640 1 -- 172.21.15.44:0/3077402477 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1df80d1260 msgr2=0x7f1df80c8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:10.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.161+0000 7f1dfe0e6640 1 --2- 172.21.15.44:0/3077402477 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1df80d1260 0x7f1df80c8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:10.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.161+0000 7f1dfe0e6640 1 -- 172.21.15.44:0/3077402477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1df80d0e80 msgr2=0x7f1df80c9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:10.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.161+0000 7f1dfe0e6640 1 --2- 172.21.15.44:0/3077402477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1df80d0e80 0x7f1df80c9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:10.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.161+0000 7f1dfe0e6640 1 -- 172.21.15.44:0/3077402477 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df80c7c10 con 0x7f1df80cc8b0 2024-04-22T06:31:10.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.161+0000 7f1dfd8e5640 1 --2- 172.21.15.44:0/3077402477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1df80d0e80 0x7f1df80c9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:31:10.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.161+0000 7f1dfe0e6640 1 --2- 172.21.15.44:0/3077402477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1df80cc8b0 0x7f1df8153d30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1df402ba50 tx=0x7f1df4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:10.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.161+0000 7f1deeffd640 1 -- 172.21.15.44:0/3077402477 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df4047070 con 0x7f1df80cc8b0 2024-04-22T06:31:10.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.161+0000 7f1deeffd640 1 -- 172.21.15.44:0/3077402477 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1df4037c40 con 0x7f1df80cc8b0 2024-04-22T06:31:10.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.161+0000 7f1e0034a640 1 -- 172.21.15.44:0/3077402477 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1df81f5500 con 0x7f1df80cc8b0 2024-04-22T06:31:10.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.161+0000 7f1deeffd640 1 -- 172.21.15.44:0/3077402477 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df4042480 con 0x7f1df80cc8b0 2024-04-22T06:31:10.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.161+0000 7f1e0034a640 1 -- 172.21.15.44:0/3077402477 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1df81f4740 con 0x7f1df80cc8b0 2024-04-22T06:31:10.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.162+0000 7f1e0034a640 1 -- 172.21.15.44:0/3077402477 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1df8068cc0 con 0x7f1df80cc8b0 2024-04-22T06:31:10.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.163+0000 7f1deeffd640 1 -- 172.21.15.44:0/3077402477 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1df404a070 con 0x7f1df80cc8b0 2024-04-22T06:31:10.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.163+0000 7f1deeffd640 1 --2- 172.21.15.44:0/3077402477 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f1ddc06c8f0 0x7f1ddc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:10.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.163+0000 7f1deeffd640 1 -- 172.21.15.44:0/3077402477 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7f1df40ae030 con 0x7f1df80cc8b0 2024-04-22T06:31:10.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.163+0000 7f1dfd8e5640 1 --2- 172.21.15.44:0/3077402477 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f1ddc06c8f0 0x7f1ddc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:10.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.164+0000 7f1dfd8e5640 1 --2- 172.21.15.44:0/3077402477 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f1ddc06c8f0 0x7f1ddc06edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1df80758f0 tx=0x7f1de8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:10.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.168+0000 7f1deeffd640 1 -- 172.21.15.44:0/3077402477 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1df4033030 con 0x7f1df80cc8b0 2024-04-22T06:31:10.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:10.352+0000 7f1e0034a640 1 -- 172.21.15.44:0/3077402477 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f1df814b0a0 con 0x7f1df80cc8b0 2024-04-22T06:31:11.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.025+0000 7f1deeffd640 1 -- 172.21.15.44:0/3077402477 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) ==== 169+0+0 (secure 0 0 0) 0x7f1df407c2f0 con 0x7f1df80cc8b0 2024-04-22T06:31:11.024 INFO:teuthology.orchestra.run.smithi044.stderr:set pool 4 allow_ec_overwrites to true 2024-04-22T06:31:11.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.027+0000 7f1e0034a640 1 -- 172.21.15.44:0/3077402477 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f1ddc06c8f0 msgr2=0x7f1ddc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:11.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.027+0000 7f1e0034a640 1 --2- 172.21.15.44:0/3077402477 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f1ddc06c8f0 0x7f1ddc06edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1df80758f0 tx=0x7f1de8008040 comp rx=0 tx=0).stop 2024-04-22T06:31:11.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.027+0000 7f1e0034a640 1 -- 172.21.15.44:0/3077402477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1df80cc8b0 msgr2=0x7f1df8153d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:11.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.027+0000 7f1e0034a640 1 --2- 172.21.15.44:0/3077402477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1df80cc8b0 0x7f1df8153d30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1df402ba50 tx=0x7f1df4004730 comp rx=0 tx=0).stop 2024-04-22T06:31:11.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.028+0000 7f1e0034a640 1 -- 172.21.15.44:0/3077402477 shutdown_connections 2024-04-22T06:31:11.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.028+0000 7f1e0034a640 1 --2- 172.21.15.44:0/3077402477 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f1ddc06c8f0 0x7f1ddc06edb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:11.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.028+0000 7f1e0034a640 1 --2- 172.21.15.44:0/3077402477 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1df80d1260 0x7f1df80c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:11.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.028+0000 7f1e0034a640 1 --2- 172.21.15.44:0/3077402477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1df80d0e80 0x7f1df80c9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:11.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.028+0000 7f1e0034a640 1 --2- 172.21.15.44:0/3077402477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1df80cc8b0 0x7f1df8153d30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:11.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.028+0000 7f1e0034a640 1 -- 172.21.15.44:0/3077402477 >> 172.21.15.44:0/3077402477 conn(0x7f1df8074e90 msgr2=0x7f1df8152d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:11.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.028+0000 7f1e0034a640 1 -- 172.21.15.44:0/3077402477 shutdown_connections 2024-04-22T06:31:11.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.028+0000 7f1e0034a640 1 -- 172.21.15.44:0/3077402477 wait complete. 2024-04-22T06:31:11.059 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-22T06:31:11.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.168+0000 7f106ef75640 1 Processor -- start 2024-04-22T06:31:11.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.168+0000 7f106ef75640 1 -- start start 2024-04-22T06:31:11.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.169+0000 7f106ef75640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1068154f00 0x7f10680cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:11.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.169+0000 7f106ef75640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f10681552e0 0x7f106815c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:11.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.169+0000 7f106ef75640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10680c5b30 con 0x7f106806d470 2024-04-22T06:31:11.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.169+0000 7f106ef75640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f10680c6740 con 0x7f1068154f00 2024-04-22T06:31:11.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.169+0000 7f106ef75640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f10680c5e10 con 0x7f10681552e0 2024-04-22T06:31:11.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.169+0000 7f106cd11640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1068154f00 0x7f10680cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:11.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.169+0000 7f106d512640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f106806d470 0x7f10680d0f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:56362/0 (socket says 172.21.15.44:56362) 2024-04-22T06:31:11.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.169+0000 7f106d512640 1 -- 172.21.15.44:0/3417344353 learned_addr learned my addr 172.21.15.44:0/3417344353 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:11.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.169+0000 7f1067fff640 1 --2- 172.21.15.44:0/3417344353 >> v2:172.21.15.70:3300/0 conn(0x7f10681552e0 0x7f106815c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:11.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.169+0000 7f106cd11640 1 -- 172.21.15.44:0/3417344353 >> v2:172.21.15.70:3300/0 conn(0x7f10681552e0 msgr2=0x7f106815c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:11.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.169+0000 7f106cd11640 1 --2- 172.21.15.44:0/3417344353 >> v2:172.21.15.70:3300/0 conn(0x7f10681552e0 0x7f106815c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:11.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.169+0000 7f106cd11640 1 -- 172.21.15.44:0/3417344353 >> v1:172.21.15.44:6789/0 conn(0x7f106806d470 legacy=0x7f10680d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:11.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.169+0000 7f106cd11640 1 -- 172.21.15.44:0/3417344353 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10680c60f0 con 0x7f1068154f00 2024-04-22T06:31:11.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.170+0000 7f106cd11640 1 --2- 172.21.15.44:0/3417344353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1068154f00 0x7f10680cb430 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f10680ce560 tx=0x7f106002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c891cc6395bf5d7c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:11.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.170+0000 7f10677fe640 1 -- 172.21.15.44:0/3417344353 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1060039070 con 0x7f1068154f00 2024-04-22T06:31:11.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.170+0000 7f10677fe640 1 -- 172.21.15.44:0/3417344353 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f106002cbf0 con 0x7f1068154f00 2024-04-22T06:31:11.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.170+0000 7f10677fe640 1 -- 172.21.15.44:0/3417344353 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1060034440 con 0x7f1068154f00 2024-04-22T06:31:11.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.170+0000 7f106ef75640 1 -- 172.21.15.44:0/3417344353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1068154f00 msgr2=0x7f10680cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:11.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.170+0000 7f106ef75640 1 --2- 172.21.15.44:0/3417344353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1068154f00 0x7f10680cb430 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f10680ce560 tx=0x7f106002be40 comp rx=0 tx=0).stop 2024-04-22T06:31:11.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.171+0000 7f106ef75640 1 -- 172.21.15.44:0/3417344353 shutdown_connections 2024-04-22T06:31:11.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.171+0000 7f106ef75640 1 --2- 172.21.15.44:0/3417344353 >> v2:172.21.15.70:3300/0 conn(0x7f10681552e0 0x7f106815c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:11.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.171+0000 7f106ef75640 1 --2- 172.21.15.44:0/3417344353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1068154f00 0x7f10680cb430 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:11.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.171+0000 7f106ef75640 1 -- 172.21.15.44:0/3417344353 >> 172.21.15.44:0/3417344353 conn(0x7f10680733c0 msgr2=0x7f10680737d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:11.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.171+0000 7f106ef75640 1 -- 172.21.15.44:0/3417344353 shutdown_connections 2024-04-22T06:31:11.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.171+0000 7f106ef75640 1 -- 172.21.15.44:0/3417344353 wait complete. 2024-04-22T06:31:11.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.172+0000 7f106ef75640 1 Processor -- start 2024-04-22T06:31:11.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.172+0000 7f106ef75640 1 -- start start 2024-04-22T06:31:11.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.172+0000 7f106ef75640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f106806d470 0x7f106814ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:11.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.172+0000 7f106ef75640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1068154f00 0x7f10681f1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:11.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.172+0000 7f106ef75640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10681552e0 0x7f10681f1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:11.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.173+0000 7f106ef75640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f10680c6d50 con 0x7f106806d470 2024-04-22T06:31:11.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.173+0000 7f106ef75640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f10680c6bd0 con 0x7f10681552e0 2024-04-22T06:31:11.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.173+0000 7f106ef75640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f10680d1690 con 0x7f1068154f00 2024-04-22T06:31:11.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.173+0000 7f106cd11640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f106806d470 0x7f106814ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:11.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.173+0000 7f106cd11640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f106806d470 0x7f106814ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60648/0 (socket says 172.21.15.44:60648) 2024-04-22T06:31:11.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.173+0000 7f106cd11640 1 -- 172.21.15.44:0/1263329884 learned_addr learned my addr 172.21.15.44:0/1263329884 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:11.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.173+0000 7f106d512640 1 --2- 172.21.15.44:0/1263329884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10681552e0 0x7f10681f1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:11.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.173+0000 7f1067fff640 1 --2- 172.21.15.44:0/1263329884 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1068154f00 0x7f10681f1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:11.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.173+0000 7f106cd11640 1 -- 172.21.15.44:0/1263329884 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1068154f00 msgr2=0x7f10681f1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:11.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.173+0000 7f106cd11640 1 --2- 172.21.15.44:0/1263329884 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1068154f00 0x7f10681f1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:11.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.173+0000 7f106cd11640 1 -- 172.21.15.44:0/1263329884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10681552e0 msgr2=0x7f10681f1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:11.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.173+0000 7f106cd11640 1 --2- 172.21.15.44:0/1263329884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10681552e0 0x7f10681f1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:11.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.173+0000 7f106cd11640 1 -- 172.21.15.44:0/1263329884 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10681f43b0 con 0x7f106806d470 2024-04-22T06:31:11.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.173+0000 7f106d512640 1 --2- 172.21.15.44:0/1263329884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10681552e0 0x7f10681f1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:31:11.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.173+0000 7f106cd11640 1 --2- 172.21.15.44:0/1263329884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f106806d470 0x7f106814ca10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1060037040 tx=0x7f1060004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:11.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.173+0000 7f10657fa640 1 -- 172.21.15.44:0/1263329884 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1060039050 con 0x7f106806d470 2024-04-22T06:31:11.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.173+0000 7f10657fa640 1 -- 172.21.15.44:0/1263329884 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1060045070 con 0x7f106806d470 2024-04-22T06:31:11.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.173+0000 7f10657fa640 1 -- 172.21.15.44:0/1263329884 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106003e850 con 0x7f106806d470 2024-04-22T06:31:11.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.173+0000 7f106ef75640 1 -- 172.21.15.44:0/1263329884 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10681f4690 con 0x7f106806d470 2024-04-22T06:31:11.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.173+0000 7f106ef75640 1 -- 172.21.15.44:0/1263329884 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10681f4c50 con 0x7f106806d470 2024-04-22T06:31:11.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.174+0000 7f106ef75640 1 -- 172.21.15.44:0/1263329884 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1068068cc0 con 0x7f106806d470 2024-04-22T06:31:11.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.175+0000 7f10657fa640 1 -- 172.21.15.44:0/1263329884 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f106003e9f0 con 0x7f106806d470 2024-04-22T06:31:11.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.175+0000 7f10657fa640 1 --2- 172.21.15.44:0/1263329884 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f104806c8f0 0x7f104806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:11.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.175+0000 7f10657fa640 1 -- 172.21.15.44:0/1263329884 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6500+0+0 (secure 0 0 0) 0x7f10600aec60 con 0x7f106806d470 2024-04-22T06:31:11.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.175+0000 7f1067fff640 1 --2- 172.21.15.44:0/1263329884 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f104806c8f0 0x7f104806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:11.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.176+0000 7f1067fff640 1 --2- 172.21.15.44:0/1263329884 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f104806c8f0 0x7f104806edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f10680d00b0 tx=0x7f1050008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:11.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.180+0000 7f10657fa640 1 -- 172.21.15.44:0/1263329884 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f106007ce10 con 0x7f106806d470 2024-04-22T06:31:11.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:11.405+0000 7f106ef75640 1 -- 172.21.15.44:0/1263329884 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f10680caee0 con 0x7f106806d470 2024-04-22T06:31:12.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.026+0000 7f10657fa640 1 -- 172.21.15.44:0/1263329884 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v5) ==== 137+0+0 (secure 0 0 0) 0x7f106007c710 con 0x7f106806d470 2024-04-22T06:31:12.025 INFO:teuthology.orchestra.run.smithi044.stderr:added data pool 4 to fsmap 2024-04-22T06:31:12.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.029+0000 7f106ef75640 1 -- 172.21.15.44:0/1263329884 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f104806c8f0 msgr2=0x7f104806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:12.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.029+0000 7f106ef75640 1 --2- 172.21.15.44:0/1263329884 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f104806c8f0 0x7f104806edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f10680d00b0 tx=0x7f1050008040 comp rx=0 tx=0).stop 2024-04-22T06:31:12.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.029+0000 7f106ef75640 1 -- 172.21.15.44:0/1263329884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f106806d470 msgr2=0x7f106814ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:12.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.029+0000 7f106ef75640 1 --2- 172.21.15.44:0/1263329884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f106806d470 0x7f106814ca10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1060037040 tx=0x7f1060004a40 comp rx=0 tx=0).stop 2024-04-22T06:31:12.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.029+0000 7f106ef75640 1 -- 172.21.15.44:0/1263329884 shutdown_connections 2024-04-22T06:31:12.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.029+0000 7f106ef75640 1 --2- 172.21.15.44:0/1263329884 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f104806c8f0 0x7f104806edb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:12.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.029+0000 7f106ef75640 1 --2- 172.21.15.44:0/1263329884 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1068154f00 0x7f10681f1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:12.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.029+0000 7f106ef75640 1 --2- 172.21.15.44:0/1263329884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10681552e0 0x7f10681f1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:12.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.029+0000 7f106ef75640 1 --2- 172.21.15.44:0/1263329884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f106806d470 0x7f106814ca10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:12.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.029+0000 7f106ef75640 1 -- 172.21.15.44:0/1263329884 >> 172.21.15.44:0/1263329884 conn(0x7f10680733c0 msgr2=0x7f1068150660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:12.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.029+0000 7f106ef75640 1 -- 172.21.15.44:0/1263329884 shutdown_connections 2024-04-22T06:31:12.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.029+0000 7f106ef75640 1 -- 172.21.15.44:0/1263329884 wait complete. 2024-04-22T06:31:12.042 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T06:31:12.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.150+0000 7f4e85a62640 1 Processor -- start 2024-04-22T06:31:12.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.150+0000 7f4e85a62640 1 -- start start 2024-04-22T06:31:12.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.151+0000 7f4e85a62640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4e800c4ba0 0x7f4e80158890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:12.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.151+0000 7f4e85a62640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4e800c4f80 0x7f4e800c53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:12.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.151+0000 7f4e85a62640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f4e80151940 0x7f4e80151d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:12.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.151+0000 7f4e85a62640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f4e800c3de0 con 0x7f4e80151940 2024-04-22T06:31:12.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.151+0000 7f4e85a62640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f4e80158dd0 con 0x7f4e800c4ba0 2024-04-22T06:31:12.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.151+0000 7f4e85a62640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f4e800c5a80 con 0x7f4e800c4f80 2024-04-22T06:31:12.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.151+0000 7f4e7ffff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f4e80151940 0x7f4e80151d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:12.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.151+0000 7f4e7ffff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f4e80151940 0x7f4e80151d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59516/0 (socket says 172.21.15.44:59516) 2024-04-22T06:31:12.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.151+0000 7f4e7ffff640 1 -- 172.21.15.44:0/3503190728 learned_addr learned my addr 172.21.15.44:0/3503190728 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:12.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.152+0000 7f4e7f7fe640 1 --2- 172.21.15.44:0/3503190728 >> v2:172.21.15.70:3300/0 conn(0x7f4e800c4ba0 0x7f4e80158890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:12.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.152+0000 7f4e7effd640 1 --2- 172.21.15.44:0/3503190728 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4e800c4f80 0x7f4e800c53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:12.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.152+0000 7f4e7ffff640 1 -- 172.21.15.44:0/3503190728 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4e800c4f80 msgr2=0x7f4e800c53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:12.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.152+0000 7f4e7ffff640 1 --2- 172.21.15.44:0/3503190728 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4e800c4f80 0x7f4e800c53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:12.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.152+0000 7f4e7ffff640 1 -- 172.21.15.44:0/3503190728 >> v2:172.21.15.70:3300/0 conn(0x7f4e800c4ba0 msgr2=0x7f4e80158890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:12.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.152+0000 7f4e7ffff640 1 --2- 172.21.15.44:0/3503190728 >> v2:172.21.15.70:3300/0 conn(0x7f4e800c4ba0 0x7f4e80158890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:12.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.152+0000 7f4e7ffff640 1 -- 172.21.15.44:0/3503190728 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e801522b0 con 0x7f4e80151940 2024-04-22T06:31:12.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.152+0000 7f4e7f7fe640 1 --2- 172.21.15.44:0/3503190728 >> v2:172.21.15.70:3300/0 conn(0x7f4e800c4ba0 0x7f4e80158890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:31:12.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.152+0000 7f4e7ffff640 1 --2- 172.21.15.44:0/3503190728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e80151940 0x7f4e80151d70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4e80075af0 tx=0x7f4e7802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac8995f247c077c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:12.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.153+0000 7f4e7e7fc640 1 -- 172.21.15.44:0/3503190728 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7802ca70 con 0x7f4e80151940 2024-04-22T06:31:12.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.153+0000 7f4e7e7fc640 1 -- 172.21.15.44:0/3503190728 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e7802cc10 con 0x7f4e80151940 2024-04-22T06:31:12.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.153+0000 7f4e7e7fc640 1 -- 172.21.15.44:0/3503190728 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e780343f0 con 0x7f4e80151940 2024-04-22T06:31:12.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.153+0000 7f4e85a62640 1 -- 172.21.15.44:0/3503190728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e80151940 msgr2=0x7f4e80151d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:12.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.153+0000 7f4e85a62640 1 --2- 172.21.15.44:0/3503190728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e80151940 0x7f4e80151d70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4e80075af0 tx=0x7f4e7802bed0 comp rx=0 tx=0).stop 2024-04-22T06:31:12.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.153+0000 7f4e85a62640 1 -- 172.21.15.44:0/3503190728 shutdown_connections 2024-04-22T06:31:12.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.153+0000 7f4e85a62640 1 --2- 172.21.15.44:0/3503190728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e80151940 0x7f4e80151d70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:12.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.153+0000 7f4e85a62640 1 --2- 172.21.15.44:0/3503190728 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4e800c4f80 0x7f4e800c53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:12.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.153+0000 7f4e85a62640 1 --2- 172.21.15.44:0/3503190728 >> v2:172.21.15.70:3300/0 conn(0x7f4e800c4ba0 0x7f4e80158890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:12.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.153+0000 7f4e85a62640 1 -- 172.21.15.44:0/3503190728 >> 172.21.15.44:0/3503190728 conn(0x7f4e80074cb0 msgr2=0x7f4e800750c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:12.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.153+0000 7f4e85a62640 1 -- 172.21.15.44:0/3503190728 shutdown_connections 2024-04-22T06:31:12.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.153+0000 7f4e85a62640 1 -- 172.21.15.44:0/3503190728 wait complete. 2024-04-22T06:31:12.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.154+0000 7f4e85a62640 1 Processor -- start 2024-04-22T06:31:12.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.155+0000 7f4e85a62640 1 -- start start 2024-04-22T06:31:12.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.155+0000 7f4e85a62640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4e800c4ba0 0x7f4e8014ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:12.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.155+0000 7f4e85a62640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e800c4f80 0x7f4e80073c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:12.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.155+0000 7f4e85a62640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e80151940 0x7f4e801493c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:12.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.155+0000 7f4e85a62640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4e800c5fb0 con 0x7f4e800c4f80 2024-04-22T06:31:12.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.155+0000 7f4e85a62640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4e800ca3c0 con 0x7f4e80151940 2024-04-22T06:31:12.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.155+0000 7f4e85a62640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f4e800c6290 con 0x7f4e800c4ba0 2024-04-22T06:31:12.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.155+0000 7f4e7effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e800c4f80 0x7f4e80073c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:12.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.155+0000 7f4e7effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e800c4f80 0x7f4e80073c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59518/0 (socket says 172.21.15.44:59518) 2024-04-22T06:31:12.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.155+0000 7f4e7effd640 1 -- 172.21.15.44:0/3069687144 learned_addr learned my addr 172.21.15.44:0/3069687144 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:12.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.155+0000 7f4e7f7fe640 1 --2- 172.21.15.44:0/3069687144 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4e800c4ba0 0x7f4e8014ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:12.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.155+0000 7f4e7ffff640 1 --2- 172.21.15.44:0/3069687144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e80151940 0x7f4e801493c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:12.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.156+0000 7f4e7effd640 1 -- 172.21.15.44:0/3069687144 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4e800c4ba0 msgr2=0x7f4e8014ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:12.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.156+0000 7f4e7effd640 1 --2- 172.21.15.44:0/3069687144 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4e800c4ba0 0x7f4e8014ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:12.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.156+0000 7f4e7effd640 1 -- 172.21.15.44:0/3069687144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e80151940 msgr2=0x7f4e801493c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:12.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.156+0000 7f4e7effd640 1 --2- 172.21.15.44:0/3069687144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e80151940 0x7f4e801493c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:12.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.156+0000 7f4e7effd640 1 -- 172.21.15.44:0/3069687144 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e8014f1a0 con 0x7f4e800c4f80 2024-04-22T06:31:12.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.156+0000 7f4e7ffff640 1 --2- 172.21.15.44:0/3069687144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e80151940 0x7f4e801493c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:31:12.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.156+0000 7f4e7f7fe640 1 --2- 172.21.15.44:0/3069687144 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4e800c4ba0 0x7f4e8014ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:31:12.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.156+0000 7f4e7effd640 1 --2- 172.21.15.44:0/3069687144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e800c4f80 0x7f4e80073c70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4e7400c820 tx=0x7f4e7400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:12.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.156+0000 7f4e67fff640 1 -- 172.21.15.44:0/3069687144 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e74015070 con 0x7f4e800c4f80 2024-04-22T06:31:12.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.156+0000 7f4e67fff640 1 -- 172.21.15.44:0/3069687144 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e74004240 con 0x7f4e800c4f80 2024-04-22T06:31:12.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.156+0000 7f4e85a62640 1 -- 172.21.15.44:0/3069687144 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e801591e0 con 0x7f4e800c4f80 2024-04-22T06:31:12.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.156+0000 7f4e67fff640 1 -- 172.21.15.44:0/3069687144 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7400ce10 con 0x7f4e800c4f80 2024-04-22T06:31:12.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.156+0000 7f4e85a62640 1 -- 172.21.15.44:0/3069687144 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e801f49b0 con 0x7f4e800c4f80 2024-04-22T06:31:12.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.157+0000 7f4e85a62640 1 -- 172.21.15.44:0/3069687144 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e80068cc0 con 0x7f4e800c4f80 2024-04-22T06:31:12.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.158+0000 7f4e67fff640 1 -- 172.21.15.44:0/3069687144 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4e74007830 con 0x7f4e800c4f80 2024-04-22T06:31:12.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.158+0000 7f4e67fff640 1 --2- 172.21.15.44:0/3069687144 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f4e6c06c8f0 0x7f4e6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:12.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.158+0000 7f4e67fff640 1 -- 172.21.15.44:0/3069687144 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4e7408bdb0 con 0x7f4e800c4f80 2024-04-22T06:31:12.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.158+0000 7f4e7f7fe640 1 --2- 172.21.15.44:0/3069687144 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f4e6c06c8f0 0x7f4e6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:12.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.159+0000 7f4e7f7fe640 1 --2- 172.21.15.44:0/3069687144 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f4e6c06c8f0 0x7f4e6c06edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4e70006b10 tx=0x7f4e70008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:12.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.163+0000 7f4e67fff640 1 -- 172.21.15.44:0/3069687144 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4e74059e10 con 0x7f4e800c4f80 2024-04-22T06:31:12.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.385+0000 7f4e85a62640 1 -- 172.21.15.44:0/3069687144 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e800cb670 con 0x7f4e800c4f80 2024-04-22T06:31:12.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.385+0000 7f4e67fff640 1 -- 172.21.15.44:0/3069687144 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+6819 (secure 0 0 0) 0x7f4e74059710 con 0x7f4e800c4f80 2024-04-22T06:31:12.385 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:31:12.385 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/1779271860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":1779271860},{"type":"v1","addr":"172.21.15.70:6836","nonce":1779271860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1497302315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1497302315},{"type":"v1","addr":"172.21.15.44:6834","nonce":1497302315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/2383621014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2383621014},{"type":"v1","addr":"172.21.15.44:6835","nonce":2383621014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4293,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/915960587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":915960587},{"type":"v1","addr":"172.21.15.44:6837","nonce":915960587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4305,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6838/924955785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":924955785},{"type":"v1","addr":"172.21.15.70:6838","nonce":924955785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T06:31:06.356312+0000","modified":"2024-04-22T06:31:12.025959+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4308},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4308":{"gid":4308,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.70:6839/2915150040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6837","nonce":2915150040},{"type":"v1","addr":"172.21.15.70:6839","nonce":2915150040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4308,"qdb_cluster":[4308]},"id":1}]} 2024-04-22T06:31:12.385 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 6 2024-04-22T06:31:12.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.387+0000 7f4e85a62640 1 -- 172.21.15.44:0/3069687144 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f4e6c06c8f0 msgr2=0x7f4e6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:12.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.387+0000 7f4e85a62640 1 --2- 172.21.15.44:0/3069687144 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f4e6c06c8f0 0x7f4e6c06edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4e70006b10 tx=0x7f4e70008040 comp rx=0 tx=0).stop 2024-04-22T06:31:12.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.387+0000 7f4e85a62640 1 -- 172.21.15.44:0/3069687144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e800c4f80 msgr2=0x7f4e80073c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:12.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.387+0000 7f4e85a62640 1 --2- 172.21.15.44:0/3069687144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e800c4f80 0x7f4e80073c70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4e7400c820 tx=0x7f4e7400cce0 comp rx=0 tx=0).stop 2024-04-22T06:31:12.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.387+0000 7f4e85a62640 1 -- 172.21.15.44:0/3069687144 shutdown_connections 2024-04-22T06:31:12.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.387+0000 7f4e85a62640 1 --2- 172.21.15.44:0/3069687144 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f4e6c06c8f0 0x7f4e6c06edb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:12.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.387+0000 7f4e85a62640 1 --2- 172.21.15.44:0/3069687144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e80151940 0x7f4e801493c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:12.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.387+0000 7f4e85a62640 1 --2- 172.21.15.44:0/3069687144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e800c4f80 0x7f4e80073c70 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:12.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.387+0000 7f4e85a62640 1 --2- 172.21.15.44:0/3069687144 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4e800c4ba0 0x7f4e8014ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:12.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.387+0000 7f4e85a62640 1 -- 172.21.15.44:0/3069687144 >> 172.21.15.44:0/3069687144 conn(0x7f4e80074cb0 msgr2=0x7f4e80150910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:12.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.388+0000 7f4e85a62640 1 -- 172.21.15.44:0/3069687144 shutdown_connections 2024-04-22T06:31:12.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.388+0000 7f4e85a62640 1 -- 172.21.15.44:0/3069687144 wait complete. 2024-04-22T06:31:12.397 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T06:31:12.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.499+0000 7f91b6f85640 1 Processor -- start 2024-04-22T06:31:12.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.499+0000 7f91b6f85640 1 -- start start 2024-04-22T06:31:12.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.500+0000 7f91b6f85640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f91b00cc480 0x7f91b015bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:12.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.500+0000 7f91b6f85640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91b00cc860 0x7f91b00cacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:12.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.500+0000 7f91b6f85640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f91b0154ca0 0x7f91b0155110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:12.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.500+0000 7f91b6f85640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f91b015f390 con 0x7f91b00cc480 2024-04-22T06:31:12.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.500+0000 7f91b6f85640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f91b015c0b0 con 0x7f91b0154ca0 2024-04-22T06:31:12.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.500+0000 7f91b6f85640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f91b00cb570 con 0x7f91b00cc860 2024-04-22T06:31:12.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.500+0000 7f91b4d21640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f91b00cc480 0x7f91b015bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:12.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.500+0000 7f91b4d21640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f91b00cc480 0x7f91b015bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59530/0 (socket says 172.21.15.44:59530) 2024-04-22T06:31:12.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.500+0000 7f91b4d21640 1 -- 172.21.15.44:0/1796314661 learned_addr learned my addr 172.21.15.44:0/1796314661 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:12.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.500+0000 7f91b5522640 1 --2- 172.21.15.44:0/1796314661 >> v2:172.21.15.70:3300/0 conn(0x7f91b0154ca0 0x7f91b0155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:12.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.500+0000 7f91a7fff640 1 --2- 172.21.15.44:0/1796314661 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91b00cc860 0x7f91b00cacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:12.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.500+0000 7f91b4d21640 1 -- 172.21.15.44:0/1796314661 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91b00cc860 msgr2=0x7f91b00cacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:12.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.500+0000 7f91b4d21640 1 --2- 172.21.15.44:0/1796314661 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91b00cc860 0x7f91b00cacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:12.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.500+0000 7f91b4d21640 1 -- 172.21.15.44:0/1796314661 >> v2:172.21.15.70:3300/0 conn(0x7f91b0154ca0 msgr2=0x7f91b0155110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:12.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.500+0000 7f91b4d21640 1 --2- 172.21.15.44:0/1796314661 >> v2:172.21.15.70:3300/0 conn(0x7f91b0154ca0 0x7f91b0155110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:12.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.500+0000 7f91b4d21640 1 -- 172.21.15.44:0/1796314661 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b015e780 con 0x7f91b00cc480 2024-04-22T06:31:12.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.500+0000 7f91b5522640 1 --2- 172.21.15.44:0/1796314661 >> v2:172.21.15.70:3300/0 conn(0x7f91b0154ca0 0x7f91b0155110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:31:12.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.500+0000 7f91a7fff640 1 --2- 172.21.15.44:0/1796314661 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91b00cc860 0x7f91b00cacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:31:12.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.501+0000 7f91b4d21640 1 --2- 172.21.15.44:0/1796314661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91b00cc480 0x7f91b015bb70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f91b0071450 tx=0x7f91ac02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=543690e643ed331c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:12.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.501+0000 7f91a77fe640 1 -- 172.21.15.44:0/1796314661 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ac035020 con 0x7f91b00cc480 2024-04-22T06:31:12.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.501+0000 7f91a77fe640 1 -- 172.21.15.44:0/1796314661 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91ac031870 con 0x7f91b00cc480 2024-04-22T06:31:12.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.501+0000 7f91a77fe640 1 -- 172.21.15.44:0/1796314661 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ac031c40 con 0x7f91b00cc480 2024-04-22T06:31:12.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.501+0000 7f91b6f85640 1 -- 172.21.15.44:0/1796314661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91b00cc480 msgr2=0x7f91b015bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:12.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.501+0000 7f91b6f85640 1 --2- 172.21.15.44:0/1796314661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91b00cc480 0x7f91b015bb70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f91b0071450 tx=0x7f91ac02b910 comp rx=0 tx=0).stop 2024-04-22T06:31:12.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.502+0000 7f91b6f85640 1 -- 172.21.15.44:0/1796314661 shutdown_connections 2024-04-22T06:31:12.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.502+0000 7f91b6f85640 1 --2- 172.21.15.44:0/1796314661 >> v2:172.21.15.70:3300/0 conn(0x7f91b0154ca0 0x7f91b0155110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:12.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.502+0000 7f91b6f85640 1 --2- 172.21.15.44:0/1796314661 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91b00cc860 0x7f91b00cacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:12.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.502+0000 7f91b6f85640 1 --2- 172.21.15.44:0/1796314661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91b00cc480 0x7f91b015bb70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:12.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.502+0000 7f91b6f85640 1 -- 172.21.15.44:0/1796314661 >> 172.21.15.44:0/1796314661 conn(0x7f91b0074cb0 msgr2=0x7f91b00750c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:12.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.502+0000 7f91b6f85640 1 -- 172.21.15.44:0/1796314661 shutdown_connections 2024-04-22T06:31:12.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.502+0000 7f91b6f85640 1 -- 172.21.15.44:0/1796314661 wait complete. 2024-04-22T06:31:12.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.503+0000 7f91b6f85640 1 Processor -- start 2024-04-22T06:31:12.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.503+0000 7f91b6f85640 1 -- start start 2024-04-22T06:31:12.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.504+0000 7f91b6f85640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91b00cc480 0x7f91b01483b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:12.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.504+0000 7f91b6f85640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91b00cc860 0x7f91b0144f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:12.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.504+0000 7f91b6f85640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91b0154ca0 0x7f91b0145450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:12.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.504+0000 7f91b6f85640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f91b015f020 con 0x7f91b0154ca0 2024-04-22T06:31:12.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.504+0000 7f91b6f85640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f91b015ed40 con 0x7f91b00cc480 2024-04-22T06:31:12.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.504+0000 7f91b6f85640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f91b00cb200 con 0x7f91b00cc860 2024-04-22T06:31:12.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.504+0000 7f91b5522640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91b0154ca0 0x7f91b0145450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:12.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.504+0000 7f91b5522640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91b0154ca0 0x7f91b0145450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59536/0 (socket says 172.21.15.44:59536) 2024-04-22T06:31:12.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.504+0000 7f91b5522640 1 -- 172.21.15.44:0/1454611461 learned_addr learned my addr 172.21.15.44:0/1454611461 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:12.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.504+0000 7f91b4d21640 1 --2- 172.21.15.44:0/1454611461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91b00cc480 0x7f91b01483b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:12.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.504+0000 7f91a7fff640 1 --2- 172.21.15.44:0/1454611461 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91b00cc860 0x7f91b0144f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:12.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.504+0000 7f91b5522640 1 -- 172.21.15.44:0/1454611461 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91b00cc860 msgr2=0x7f91b0144f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:12.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.504+0000 7f91b5522640 1 --2- 172.21.15.44:0/1454611461 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91b00cc860 0x7f91b0144f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:12.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.504+0000 7f91b5522640 1 -- 172.21.15.44:0/1454611461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91b00cc480 msgr2=0x7f91b01483b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:12.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.504+0000 7f91b5522640 1 --2- 172.21.15.44:0/1454611461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91b00cc480 0x7f91b01483b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:12.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.504+0000 7f91b5522640 1 -- 172.21.15.44:0/1454611461 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b015e2b0 con 0x7f91b0154ca0 2024-04-22T06:31:12.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.504+0000 7f91b5522640 1 --2- 172.21.15.44:0/1454611461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91b0154ca0 0x7f91b0145450 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f91a800cd50 tx=0x7f91a800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:12.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.504+0000 7f91a57fa640 1 -- 172.21.15.44:0/1454611461 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a800f680 con 0x7f91b0154ca0 2024-04-22T06:31:12.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.504+0000 7f91a57fa640 1 -- 172.21.15.44:0/1454611461 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91a800f840 con 0x7f91b0154ca0 2024-04-22T06:31:12.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.504+0000 7f91a57fa640 1 -- 172.21.15.44:0/1454611461 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a800fc30 con 0x7f91b0154ca0 2024-04-22T06:31:12.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.504+0000 7f91b6f85640 1 -- 172.21.15.44:0/1454611461 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91b0145d50 con 0x7f91b0154ca0 2024-04-22T06:31:12.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.504+0000 7f91b6f85640 1 -- 172.21.15.44:0/1454611461 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91b0146310 con 0x7f91b0154ca0 2024-04-22T06:31:12.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.505+0000 7f91b6f85640 1 -- 172.21.15.44:0/1454611461 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91b0068cc0 con 0x7f91b0154ca0 2024-04-22T06:31:12.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.506+0000 7f91a57fa640 1 -- 172.21.15.44:0/1454611461 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f91a80213f0 con 0x7f91b0154ca0 2024-04-22T06:31:12.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.506+0000 7f91a57fa640 1 --2- 172.21.15.44:0/1454611461 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f919c06c8f0 0x7f919c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:12.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.506+0000 7f91a57fa640 1 -- 172.21.15.44:0/1454611461 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f91a808cd30 con 0x7f91b0154ca0 2024-04-22T06:31:12.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.506+0000 7f91b4d21640 1 --2- 172.21.15.44:0/1454611461 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f919c06c8f0 0x7f919c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:12.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.507+0000 7f91b4d21640 1 --2- 172.21.15.44:0/1454611461 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f919c06c8f0 0x7f919c06edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f91ac036000 tx=0x7f91ac008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:12.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.511+0000 7f91a57fa640 1 -- 172.21.15.44:0/1454611461 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f91a8014030 con 0x7f91b0154ca0 2024-04-22T06:31:12.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.699+0000 7f91b6f85640 1 -- 172.21.15.44:0/1454611461 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f91b015a430 con 0x7f91b0154ca0 2024-04-22T06:31:12.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.700+0000 7f91a57fa640 1 -- 172.21.15.44:0/1454611461 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18466 (secure 0 0 0) 0x7f91a805ad90 con 0x7f91b0154ca0 2024-04-22T06:31:12.699 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:31:12.699 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"fsid":"9b0a34ca-403d-4513-8a66-d3749ba559aa","created":"2024-04-22T06:30:42.334581+0000","modified":"2024-04-22T06:31:11.406256+0000","last_up_change":"2024-04-22T06:30:52.942257+0000","last_in_change":"2024-04-22T06:30:44.467727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T06:30:54.560256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T06:31:04.612387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T06:31:05.312124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-22T06:31:09.025550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b4a6692-53c9-472d-b2f3-1dd29a6679ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6817","nonce":2220268493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6819","nonce":2220268493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6823","nonce":2220268493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6821","nonce":2220268493}]},"public_addr":"172.21.15.44:6817/2220268493","cluster_addr":"172.21.15.44:6819/2220268493","heartbeat_back_addr":"172.21.15.44:6823/2220268493","heartbeat_front_addr":"172.21.15.44:6821/2220268493","state":["exists","up"]},{"osd":1,"uuid":"70a6370a-7c0a-4d64-a0e4-b2e726b4f291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6809","nonce":1278678977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6811","nonce":1278678977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6815","nonce":1278678977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6813","nonce":1278678977}]},"public_addr":"172.21.15.44:6809/1278678977","cluster_addr":"172.21.15.44:6811/1278678977","heartbeat_back_addr":"172.21.15.44:6815/1278678977","heartbeat_front_addr":"172.21.15.44:6813/1278678977","state":["exists","up"]},{"osd":2,"uuid":"f2e51a62-ca91-470d-bd5a-5710c61deee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6801","nonce":2544087903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6803","nonce":2544087903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6807","nonce":2544087903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6805","nonce":2544087903}]},"public_addr":"172.21.15.44:6801/2544087903","cluster_addr":"172.21.15.44:6803/2544087903","heartbeat_back_addr":"172.21.15.44:6807/2544087903","heartbeat_front_addr":"172.21.15.44:6805/2544087903","state":["exists","up"]},{"osd":3,"uuid":"bc514203-9af9-4f1f-b18e-765bdd5de6e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6825","nonce":1692928109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6827","nonce":1692928109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6831","nonce":1692928109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6829","nonce":1692928109}]},"public_addr":"172.21.15.44:6825/1692928109","cluster_addr":"172.21.15.44:6827/1692928109","heartbeat_back_addr":"172.21.15.44:6831/1692928109","heartbeat_front_addr":"172.21.15.44:6829/1692928109","state":["exists","up"]},{"osd":4,"uuid":"357c439a-6e08-4c19-ac18-b68e6c890497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6820","nonce":3802577427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6824","nonce":3802577427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6831","nonce":3802577427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6828","nonce":3802577427}]},"public_addr":"172.21.15.70:6820/3802577427","cluster_addr":"172.21.15.70:6824/3802577427","heartbeat_back_addr":"172.21.15.70:6831/3802577427","heartbeat_front_addr":"172.21.15.70:6828/3802577427","state":["exists","up"]},{"osd":5,"uuid":"952fb88f-d56e-4305-bfa8-d32674332dfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6801","nonce":664757345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6803","nonce":664757345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6807","nonce":664757345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6805","nonce":664757345}]},"public_addr":"172.21.15.70:6801/664757345","cluster_addr":"172.21.15.70:6803/664757345","heartbeat_back_addr":"172.21.15.70:6807/664757345","heartbeat_front_addr":"172.21.15.70:6805/664757345","state":["exists","up"]},{"osd":6,"uuid":"74399b3d-fac2-45ab-9172-2cd95b02450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6817","nonce":1623253482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6821","nonce":1623253482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6827","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6829","nonce":1623253482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6825","nonce":1623253482}]},"public_addr":"172.21.15.70:6817/1623253482","cluster_addr":"172.21.15.70:6821/1623253482","heartbeat_back_addr":"172.21.15.70:6829/1623253482","heartbeat_front_addr":"172.21.15.70:6825/1623253482","state":["exists","up"]},{"osd":7,"uuid":"8f35508d-6fcc-43ff-8ee7-0498a7ab3b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6809","nonce":1551180300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6811","nonce":1551180300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6815","nonce":1551180300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6813","nonce":1551180300}]},"public_addr":"172.21.15.70:6809/1551180300","cluster_addr":"172.21.15.70:6811/1551180300","heartbeat_back_addr":"172.21.15.70:6815/1551180300","heartbeat_front_addr":"172.21.15.70:6813/1551180300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.612502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.653756+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.521169+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.649955+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.640231+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.686638+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.642199+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.700360+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T06:31:12.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.702+0000 7f91b6f85640 1 -- 172.21.15.44:0/1454611461 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f919c06c8f0 msgr2=0x7f919c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:12.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.702+0000 7f91b6f85640 1 --2- 172.21.15.44:0/1454611461 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f919c06c8f0 0x7f919c06edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f91ac036000 tx=0x7f91ac008030 comp rx=0 tx=0).stop 2024-04-22T06:31:12.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.702+0000 7f91b6f85640 1 -- 172.21.15.44:0/1454611461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91b0154ca0 msgr2=0x7f91b0145450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:12.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.702+0000 7f91b6f85640 1 --2- 172.21.15.44:0/1454611461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91b0154ca0 0x7f91b0145450 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f91a800cd50 tx=0x7f91a800a650 comp rx=0 tx=0).stop 2024-04-22T06:31:12.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.702+0000 7f91b6f85640 1 -- 172.21.15.44:0/1454611461 shutdown_connections 2024-04-22T06:31:12.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.702+0000 7f91b6f85640 1 --2- 172.21.15.44:0/1454611461 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f919c06c8f0 0x7f919c06edb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:12.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.702+0000 7f91b6f85640 1 --2- 172.21.15.44:0/1454611461 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91b00cc860 0x7f91b0144f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:12.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.702+0000 7f91b6f85640 1 --2- 172.21.15.44:0/1454611461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91b0154ca0 0x7f91b0145450 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:12.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.702+0000 7f91b6f85640 1 --2- 172.21.15.44:0/1454611461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91b00cc480 0x7f91b01483b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:12.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.702+0000 7f91b6f85640 1 -- 172.21.15.44:0/1454611461 >> 172.21.15.44:0/1454611461 conn(0x7f91b0074cb0 msgr2=0x7f91b014f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:12.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.702+0000 7f91b6f85640 1 -- 172.21.15.44:0/1454611461 shutdown_connections 2024-04-22T06:31:12.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.702+0000 7f91b6f85640 1 -- 172.21.15.44:0/1454611461 wait complete. 2024-04-22T06:31:12.712 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T06:31:12.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.815+0000 7fb076474640 1 Processor -- start 2024-04-22T06:31:12.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.815+0000 7fb076474640 1 -- start start 2024-04-22T06:31:12.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.816+0000 7fb076474640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb070155240 0x7fb07015f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:12.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.816+0000 7fb076474640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb0700cb320 0x7fb0700d0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:12.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.816+0000 7fb076474640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb07015b200 con 0x7fb0700d1400 2024-04-22T06:31:12.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.816+0000 7fb076474640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fb07015f610 con 0x7fb070155240 2024-04-22T06:31:12.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.816+0000 7fb076474640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fb07015c060 con 0x7fb0700cb320 2024-04-22T06:31:12.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.816+0000 7fb06ffff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb070155240 0x7fb07015f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:12.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.816+0000 7fb074a11640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fb0700d1400 0x7fb0700cc480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:43094/0 (socket says 172.21.15.44:43094) 2024-04-22T06:31:12.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.816+0000 7fb074a11640 1 -- 172.21.15.44:0/1303652982 learned_addr learned my addr 172.21.15.44:0/1303652982 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:12.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.816+0000 7fb06effd640 1 -- 172.21.15.44:0/1303652982 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3568251852 0 0) 0x7fb07015b200 con 0x7fb0700d1400 2024-04-22T06:31:12.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.816+0000 7fb06f7fe640 1 --2- 172.21.15.44:0/1303652982 >> v2:172.21.15.70:3300/0 conn(0x7fb0700cb320 0x7fb0700d0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:12.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.816+0000 7fb06effd640 1 -- 172.21.15.44:0/1303652982 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb058003520 con 0x7fb0700d1400 2024-04-22T06:31:12.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.816+0000 7fb06ffff640 1 -- 172.21.15.44:0/1303652982 >> v2:172.21.15.70:3300/0 conn(0x7fb0700cb320 msgr2=0x7fb0700d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:12.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.816+0000 7fb06ffff640 1 --2- 172.21.15.44:0/1303652982 >> v2:172.21.15.70:3300/0 conn(0x7fb0700cb320 0x7fb0700d0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:12.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.816+0000 7fb06ffff640 1 -- 172.21.15.44:0/1303652982 >> v1:172.21.15.44:6789/0 conn(0x7fb0700d1400 legacy=0x7fb0700cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:12.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.816+0000 7fb06ffff640 1 -- 172.21.15.44:0/1303652982 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb07015b4e0 con 0x7fb070155240 2024-04-22T06:31:12.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.816+0000 7fb06f7fe640 1 --2- 172.21.15.44:0/1303652982 >> v2:172.21.15.70:3300/0 conn(0x7fb0700cb320 0x7fb0700d0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:31:12.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.817+0000 7fb06ffff640 1 --2- 172.21.15.44:0/1303652982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb070155240 0x7fb07015f0d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb0700c7830 tx=0x7fb06402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f6447b863b086af server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:12.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.817+0000 7fb06effd640 1 -- 172.21.15.44:0/1303652982 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb064035020 con 0x7fb070155240 2024-04-22T06:31:12.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.817+0000 7fb06effd640 1 -- 172.21.15.44:0/1303652982 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb064031830 con 0x7fb070155240 2024-04-22T06:31:12.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.817+0000 7fb06effd640 1 -- 172.21.15.44:0/1303652982 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb064031c00 con 0x7fb070155240 2024-04-22T06:31:12.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.817+0000 7fb076474640 1 -- 172.21.15.44:0/1303652982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb070155240 msgr2=0x7fb07015f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:12.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.817+0000 7fb076474640 1 --2- 172.21.15.44:0/1303652982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb070155240 0x7fb07015f0d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb0700c7830 tx=0x7fb06402b8d0 comp rx=0 tx=0).stop 2024-04-22T06:31:12.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.818+0000 7fb076474640 1 -- 172.21.15.44:0/1303652982 shutdown_connections 2024-04-22T06:31:12.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.818+0000 7fb076474640 1 --2- 172.21.15.44:0/1303652982 >> v2:172.21.15.70:3300/0 conn(0x7fb0700cb320 0x7fb0700d0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:12.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.818+0000 7fb076474640 1 --2- 172.21.15.44:0/1303652982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb070155240 0x7fb07015f0d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:12.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.818+0000 7fb076474640 1 -- 172.21.15.44:0/1303652982 >> 172.21.15.44:0/1303652982 conn(0x7fb0700750d0 msgr2=0x7fb0700754e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:12.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.818+0000 7fb076474640 1 -- 172.21.15.44:0/1303652982 shutdown_connections 2024-04-22T06:31:12.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.818+0000 7fb076474640 1 -- 172.21.15.44:0/1303652982 wait complete. 2024-04-22T06:31:12.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.819+0000 7fb076474640 1 Processor -- start 2024-04-22T06:31:12.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.819+0000 7fb076474640 1 -- start start 2024-04-22T06:31:12.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.820+0000 7fb076474640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0700cb320 0x7fb07015d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:12.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.820+0000 7fb076474640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb0700d1400 0x7fb0700cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:12.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.820+0000 7fb076474640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb070155240 0x7fb0700d0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:12.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.820+0000 7fb076474640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb07015bd80 con 0x7fb070155240 2024-04-22T06:31:12.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.820+0000 7fb076474640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb07015baa0 con 0x7fb0700cb320 2024-04-22T06:31:12.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.820+0000 7fb076474640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb070155620 con 0x7fb0700d1400 2024-04-22T06:31:12.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.820+0000 7fb074a11640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb070155240 0x7fb0700d0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:12.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.820+0000 7fb074a11640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb070155240 0x7fb0700d0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59566/0 (socket says 172.21.15.44:59566) 2024-04-22T06:31:12.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.820+0000 7fb074a11640 1 -- 172.21.15.44:0/118118546 learned_addr learned my addr 172.21.15.44:0/118118546 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:12.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.820+0000 7fb06ffff640 1 --2- 172.21.15.44:0/118118546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0700cb320 0x7fb07015d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:12.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.820+0000 7fb074a11640 1 -- 172.21.15.44:0/118118546 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb0700d1400 msgr2=0x7fb0700cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:12.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.820+0000 7fb074a11640 1 --2- 172.21.15.44:0/118118546 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb0700d1400 0x7fb0700cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:12.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.820+0000 7fb074a11640 1 -- 172.21.15.44:0/118118546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0700cb320 msgr2=0x7fb07015d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:12.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.820+0000 7fb074a11640 1 --2- 172.21.15.44:0/118118546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0700cb320 0x7fb07015d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:12.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.820+0000 7fb074a11640 1 -- 172.21.15.44:0/118118546 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0700c8610 con 0x7fb070155240 2024-04-22T06:31:12.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.820+0000 7fb074a11640 1 --2- 172.21.15.44:0/118118546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb070155240 0x7fb0700d0060 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb06800cb40 tx=0x7fb06800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:12.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.820+0000 7fb06cff9640 1 -- 172.21.15.44:0/118118546 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb068017070 con 0x7fb070155240 2024-04-22T06:31:12.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.820+0000 7fb06cff9640 1 -- 172.21.15.44:0/118118546 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb06800aca0 con 0x7fb070155240 2024-04-22T06:31:12.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.820+0000 7fb076474640 1 -- 172.21.15.44:0/118118546 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb07015dc70 con 0x7fb070155240 2024-04-22T06:31:12.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.820+0000 7fb06cff9640 1 -- 172.21.15.44:0/118118546 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb068012620 con 0x7fb070155240 2024-04-22T06:31:12.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.820+0000 7fb076474640 1 -- 172.21.15.44:0/118118546 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0700cf040 con 0x7fb070155240 2024-04-22T06:31:12.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.821+0000 7fb076474640 1 -- 172.21.15.44:0/118118546 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0700b3820 con 0x7fb070155240 2024-04-22T06:31:12.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.822+0000 7fb06cff9640 1 -- 172.21.15.44:0/118118546 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb06801a070 con 0x7fb070155240 2024-04-22T06:31:12.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.822+0000 7fb06cff9640 1 --2- 172.21.15.44:0/118118546 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb05c06c8f0 0x7fb05c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:12.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.822+0000 7fb06cff9640 1 -- 172.21.15.44:0/118118546 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb06808bcf0 con 0x7fb070155240 2024-04-22T06:31:12.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.822+0000 7fb06ffff640 1 --2- 172.21.15.44:0/118118546 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb05c06c8f0 0x7fb05c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:12.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.823+0000 7fb06ffff640 1 --2- 172.21.15.44:0/118118546 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb05c06c8f0 0x7fb05c06edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb064036000 tx=0x7fb064008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:12.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:12.827+0000 7fb06cff9640 1 -- 172.21.15.44:0/118118546 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb0700b3820 con 0x7fb070155240 2024-04-22T06:31:13.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.012+0000 7fb076474640 1 -- 172.21.15.44:0/118118546 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb070068cc0 con 0x7fb070155240 2024-04-22T06:31:13.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.013+0000 7fb06cff9640 1 -- 172.21.15.44:0/118118546 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18466 (secure 0 0 0) 0x7fb068090070 con 0x7fb070155240 2024-04-22T06:31:13.012 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:31:13.013 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"fsid":"9b0a34ca-403d-4513-8a66-d3749ba559aa","created":"2024-04-22T06:30:42.334581+0000","modified":"2024-04-22T06:31:11.406256+0000","last_up_change":"2024-04-22T06:30:52.942257+0000","last_in_change":"2024-04-22T06:30:44.467727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T06:30:54.560256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T06:31:04.612387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T06:31:05.312124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-22T06:31:09.025550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b4a6692-53c9-472d-b2f3-1dd29a6679ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6817","nonce":2220268493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6819","nonce":2220268493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6823","nonce":2220268493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6821","nonce":2220268493}]},"public_addr":"172.21.15.44:6817/2220268493","cluster_addr":"172.21.15.44:6819/2220268493","heartbeat_back_addr":"172.21.15.44:6823/2220268493","heartbeat_front_addr":"172.21.15.44:6821/2220268493","state":["exists","up"]},{"osd":1,"uuid":"70a6370a-7c0a-4d64-a0e4-b2e726b4f291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6809","nonce":1278678977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6811","nonce":1278678977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6815","nonce":1278678977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6813","nonce":1278678977}]},"public_addr":"172.21.15.44:6809/1278678977","cluster_addr":"172.21.15.44:6811/1278678977","heartbeat_back_addr":"172.21.15.44:6815/1278678977","heartbeat_front_addr":"172.21.15.44:6813/1278678977","state":["exists","up"]},{"osd":2,"uuid":"f2e51a62-ca91-470d-bd5a-5710c61deee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6801","nonce":2544087903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6803","nonce":2544087903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6807","nonce":2544087903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6805","nonce":2544087903}]},"public_addr":"172.21.15.44:6801/2544087903","cluster_addr":"172.21.15.44:6803/2544087903","heartbeat_back_addr":"172.21.15.44:6807/2544087903","heartbeat_front_addr":"172.21.15.44:6805/2544087903","state":["exists","up"]},{"osd":3,"uuid":"bc514203-9af9-4f1f-b18e-765bdd5de6e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6825","nonce":1692928109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6827","nonce":1692928109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6831","nonce":1692928109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6829","nonce":1692928109}]},"public_addr":"172.21.15.44:6825/1692928109","cluster_addr":"172.21.15.44:6827/1692928109","heartbeat_back_addr":"172.21.15.44:6831/1692928109","heartbeat_front_addr":"172.21.15.44:6829/1692928109","state":["exists","up"]},{"osd":4,"uuid":"357c439a-6e08-4c19-ac18-b68e6c890497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6820","nonce":3802577427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6824","nonce":3802577427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6831","nonce":3802577427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6828","nonce":3802577427}]},"public_addr":"172.21.15.70:6820/3802577427","cluster_addr":"172.21.15.70:6824/3802577427","heartbeat_back_addr":"172.21.15.70:6831/3802577427","heartbeat_front_addr":"172.21.15.70:6828/3802577427","state":["exists","up"]},{"osd":5,"uuid":"952fb88f-d56e-4305-bfa8-d32674332dfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6801","nonce":664757345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6803","nonce":664757345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6807","nonce":664757345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6805","nonce":664757345}]},"public_addr":"172.21.15.70:6801/664757345","cluster_addr":"172.21.15.70:6803/664757345","heartbeat_back_addr":"172.21.15.70:6807/664757345","heartbeat_front_addr":"172.21.15.70:6805/664757345","state":["exists","up"]},{"osd":6,"uuid":"74399b3d-fac2-45ab-9172-2cd95b02450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6817","nonce":1623253482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6821","nonce":1623253482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6827","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6829","nonce":1623253482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6825","nonce":1623253482}]},"public_addr":"172.21.15.70:6817/1623253482","cluster_addr":"172.21.15.70:6821/1623253482","heartbeat_back_addr":"172.21.15.70:6829/1623253482","heartbeat_front_addr":"172.21.15.70:6825/1623253482","state":["exists","up"]},{"osd":7,"uuid":"8f35508d-6fcc-43ff-8ee7-0498a7ab3b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6809","nonce":1551180300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6811","nonce":1551180300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6815","nonce":1551180300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6813","nonce":1551180300}]},"public_addr":"172.21.15.70:6809/1551180300","cluster_addr":"172.21.15.70:6811/1551180300","heartbeat_back_addr":"172.21.15.70:6815/1551180300","heartbeat_front_addr":"172.21.15.70:6813/1551180300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.612502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.653756+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.521169+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.649955+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.640231+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.686638+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.642199+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.700360+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T06:31:13.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.016+0000 7fb076474640 1 -- 172.21.15.44:0/118118546 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb05c06c8f0 msgr2=0x7fb05c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:13.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.016+0000 7fb076474640 1 --2- 172.21.15.44:0/118118546 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb05c06c8f0 0x7fb05c06edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb064036000 tx=0x7fb064008030 comp rx=0 tx=0).stop 2024-04-22T06:31:13.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.016+0000 7fb076474640 1 -- 172.21.15.44:0/118118546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb070155240 msgr2=0x7fb0700d0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:13.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.016+0000 7fb076474640 1 --2- 172.21.15.44:0/118118546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb070155240 0x7fb0700d0060 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb06800cb40 tx=0x7fb06800a060 comp rx=0 tx=0).stop 2024-04-22T06:31:13.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.016+0000 7fb076474640 1 -- 172.21.15.44:0/118118546 shutdown_connections 2024-04-22T06:31:13.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.016+0000 7fb076474640 1 --2- 172.21.15.44:0/118118546 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb05c06c8f0 0x7fb05c06edb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:13.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.016+0000 7fb076474640 1 --2- 172.21.15.44:0/118118546 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb0700d1400 0x7fb0700cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:13.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.016+0000 7fb076474640 1 --2- 172.21.15.44:0/118118546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb070155240 0x7fb0700d0060 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:13.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.016+0000 7fb076474640 1 --2- 172.21.15.44:0/118118546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0700cb320 0x7fb07015d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:13.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.016+0000 7fb076474640 1 -- 172.21.15.44:0/118118546 >> 172.21.15.44:0/118118546 conn(0x7fb0700750d0 msgr2=0x7fb07014a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:13.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.016+0000 7fb076474640 1 -- 172.21.15.44:0/118118546 shutdown_connections 2024-04-22T06:31:13.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.016+0000 7fb076474640 1 -- 172.21.15.44:0/118118546 wait complete. 2024-04-22T06:31:13.083 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T06:31:13.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.184+0000 7fccc1631640 1 Processor -- start 2024-04-22T06:31:13.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.184+0000 7fccc1631640 1 -- start start 2024-04-22T06:31:13.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.185+0000 7fccc1631640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fccbc155340 0x7fccbc0cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:13.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.185+0000 7fccc1631640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fccbc0cc310 0x7fccbc0cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:13.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.185+0000 7fccc1631640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccbc0c5b70 con 0x7fccbc15e610 2024-04-22T06:31:13.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.185+0000 7fccc1631640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fccbc15b020 con 0x7fccbc155340 2024-04-22T06:31:13.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.185+0000 7fccc1631640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fccbc155720 con 0x7fccbc0cc310 2024-04-22T06:31:13.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.185+0000 7fccbaffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fccbc155340 0x7fccbc0cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:13.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.185+0000 7fccba7fc640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fccbc15e610 0x7fccbc15b800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:43098/0 (socket says 172.21.15.44:43098) 2024-04-22T06:31:13.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.185+0000 7fccba7fc640 1 -- 172.21.15.44:0/770375690 learned_addr learned my addr 172.21.15.44:0/770375690 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:13.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.185+0000 7fccbb7fe640 1 --2- 172.21.15.44:0/770375690 >> v2:172.21.15.70:3300/0 conn(0x7fccbc0cc310 0x7fccbc0cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:13.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.185+0000 7fccbaffd640 1 -- 172.21.15.44:0/770375690 >> v2:172.21.15.70:3300/0 conn(0x7fccbc0cc310 msgr2=0x7fccbc0cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:13.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.185+0000 7fccbaffd640 1 --2- 172.21.15.44:0/770375690 >> v2:172.21.15.70:3300/0 conn(0x7fccbc0cc310 0x7fccbc0cb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:13.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.185+0000 7fccbaffd640 1 -- 172.21.15.44:0/770375690 >> v1:172.21.15.44:6789/0 conn(0x7fccbc15e610 legacy=0x7fccbc15b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:13.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.185+0000 7fccbaffd640 1 -- 172.21.15.44:0/770375690 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccbc15eb50 con 0x7fccbc155340 2024-04-22T06:31:13.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.185+0000 7fccb9ffb640 1 -- 172.21.15.44:0/770375690 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4081530899 0 0) 0x7fccbc0c5b70 con 0x7fccbc15e610 2024-04-22T06:31:13.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.186+0000 7fccbaffd640 1 --2- 172.21.15.44:0/770375690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccbc155340 0x7fccbc0cd7b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fccbc0ce030 tx=0x7fccb402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9957176de462113 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:13.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.186+0000 7fccb9ffb640 1 -- 172.21.15.44:0/770375690 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb4035020 con 0x7fccbc155340 2024-04-22T06:31:13.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.186+0000 7fccb9ffb640 1 -- 172.21.15.44:0/770375690 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccb4031830 con 0x7fccbc155340 2024-04-22T06:31:13.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.186+0000 7fccb9ffb640 1 -- 172.21.15.44:0/770375690 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb4031c00 con 0x7fccbc155340 2024-04-22T06:31:13.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.187+0000 7fccc1631640 1 -- 172.21.15.44:0/770375690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccbc155340 msgr2=0x7fccbc0cd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:13.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.187+0000 7fccc1631640 1 --2- 172.21.15.44:0/770375690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccbc155340 0x7fccbc0cd7b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fccbc0ce030 tx=0x7fccb402b8d0 comp rx=0 tx=0).stop 2024-04-22T06:31:13.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.187+0000 7fccc1631640 1 -- 172.21.15.44:0/770375690 shutdown_connections 2024-04-22T06:31:13.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.187+0000 7fccc1631640 1 --2- 172.21.15.44:0/770375690 >> v2:172.21.15.70:3300/0 conn(0x7fccbc0cc310 0x7fccbc0cb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:13.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.187+0000 7fccc1631640 1 --2- 172.21.15.44:0/770375690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccbc155340 0x7fccbc0cd7b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:13.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.187+0000 7fccc1631640 1 -- 172.21.15.44:0/770375690 >> 172.21.15.44:0/770375690 conn(0x7fccbc06c560 msgr2=0x7fccbc074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:13.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.187+0000 7fccc1631640 1 -- 172.21.15.44:0/770375690 shutdown_connections 2024-04-22T06:31:13.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.187+0000 7fccc1631640 1 -- 172.21.15.44:0/770375690 wait complete. 2024-04-22T06:31:13.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.188+0000 7fccc1631640 1 Processor -- start 2024-04-22T06:31:13.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.188+0000 7fccc1631640 1 -- start start 2024-04-22T06:31:13.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.189+0000 7fccc1631640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fccbc0cc310 0x7fccbc153e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:13.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.189+0000 7fccc1631640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccbc155340 0x7fccbc0c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:13.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.189+0000 7fccc1631640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccbc15e610 0x7fccbc0ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:13.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.189+0000 7fccc1631640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fccbc15f3f0 con 0x7fccbc155340 2024-04-22T06:31:13.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.189+0000 7fccc1631640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fccbc15f110 con 0x7fccbc15e610 2024-04-22T06:31:13.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.189+0000 7fccc1631640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fccbc15b2e0 con 0x7fccbc0cc310 2024-04-22T06:31:13.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.189+0000 7fccba7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccbc155340 0x7fccbc0c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:13.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.189+0000 7fccba7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccbc155340 0x7fccbc0c9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59586/0 (socket says 172.21.15.44:59586) 2024-04-22T06:31:13.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.189+0000 7fccba7fc640 1 -- 172.21.15.44:0/1143517793 learned_addr learned my addr 172.21.15.44:0/1143517793 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:13.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.189+0000 7fccbb7fe640 1 --2- 172.21.15.44:0/1143517793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccbc15e610 0x7fccbc0ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:13.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.189+0000 7fccbaffd640 1 --2- 172.21.15.44:0/1143517793 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fccbc0cc310 0x7fccbc153e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:13.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.189+0000 7fccba7fc640 1 -- 172.21.15.44:0/1143517793 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fccbc0cc310 msgr2=0x7fccbc153e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:13.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.189+0000 7fccba7fc640 1 --2- 172.21.15.44:0/1143517793 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fccbc0cc310 0x7fccbc153e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:13.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.189+0000 7fccba7fc640 1 -- 172.21.15.44:0/1143517793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccbc15e610 msgr2=0x7fccbc0ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:13.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.189+0000 7fccba7fc640 1 --2- 172.21.15.44:0/1143517793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccbc15e610 0x7fccbc0ca1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:13.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.189+0000 7fccba7fc640 1 -- 172.21.15.44:0/1143517793 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccbc15dc90 con 0x7fccbc155340 2024-04-22T06:31:13.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.190+0000 7fccba7fc640 1 --2- 172.21.15.44:0/1143517793 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccbc155340 0x7fccbc0c9c90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcca400c3f0 tx=0x7fcca400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:13.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.190+0000 7fccb37fe640 1 -- 172.21.15.44:0/1143517793 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca40074e0 con 0x7fccbc155340 2024-04-22T06:31:13.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.190+0000 7fccb37fe640 1 -- 172.21.15.44:0/1143517793 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcca40047e0 con 0x7fccbc155340 2024-04-22T06:31:13.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.190+0000 7fccb37fe640 1 -- 172.21.15.44:0/1143517793 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca4004c10 con 0x7fccbc155340 2024-04-22T06:31:13.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.190+0000 7fccc1631640 1 -- 172.21.15.44:0/1143517793 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccbc15df50 con 0x7fccbc155340 2024-04-22T06:31:13.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.190+0000 7fccc1631640 1 -- 172.21.15.44:0/1143517793 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccbc0b5420 con 0x7fccbc155340 2024-04-22T06:31:13.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.191+0000 7fccc1631640 1 -- 172.21.15.44:0/1143517793 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccbc068cc0 con 0x7fccbc155340 2024-04-22T06:31:13.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.191+0000 7fccb37fe640 1 -- 172.21.15.44:0/1143517793 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcca4011070 con 0x7fccbc155340 2024-04-22T06:31:13.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.192+0000 7fccb37fe640 1 --2- 172.21.15.44:0/1143517793 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fcc9c06c8f0 0x7fcc9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:13.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.192+0000 7fccb37fe640 1 -- 172.21.15.44:0/1143517793 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fcca408bb60 con 0x7fccbc155340 2024-04-22T06:31:13.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.192+0000 7fccbaffd640 1 --2- 172.21.15.44:0/1143517793 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fcc9c06c8f0 0x7fcc9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:13.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.192+0000 7fccbaffd640 1 --2- 172.21.15.44:0/1143517793 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fcc9c06c8f0 0x7fcc9c06edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fccb4036000 tx=0x7fccb4008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:13.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.196+0000 7fccb37fe640 1 -- 172.21.15.44:0/1143517793 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcca401a2b0 con 0x7fccbc155340 2024-04-22T06:31:13.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.421+0000 7fccc1631640 1 -- 172.21.15.44:0/1143517793 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fccbc0cc040 con 0x7fccbc155340 2024-04-22T06:31:13.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.422+0000 7fccb37fe640 1 -- 172.21.15.44:0/1143517793 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+6819 (secure 0 0 0) 0x7fcca4059bb0 con 0x7fccbc155340 2024-04-22T06:31:13.421 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:31:13.421 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/1779271860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":1779271860},{"type":"v1","addr":"172.21.15.70:6836","nonce":1779271860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1497302315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1497302315},{"type":"v1","addr":"172.21.15.44:6834","nonce":1497302315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/2383621014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2383621014},{"type":"v1","addr":"172.21.15.44:6835","nonce":2383621014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4293,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/915960587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":915960587},{"type":"v1","addr":"172.21.15.44:6837","nonce":915960587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4305,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6838/924955785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":924955785},{"type":"v1","addr":"172.21.15.70:6838","nonce":924955785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T06:31:06.356312+0000","modified":"2024-04-22T06:31:12.025959+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4308},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4308":{"gid":4308,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.70:6839/2915150040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6837","nonce":2915150040},{"type":"v1","addr":"172.21.15.70:6839","nonce":2915150040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4308,"qdb_cluster":[4308]},"id":1}]} 2024-04-22T06:31:13.421 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 6 2024-04-22T06:31:13.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.423+0000 7fccc1631640 1 -- 172.21.15.44:0/1143517793 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fcc9c06c8f0 msgr2=0x7fcc9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:13.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.423+0000 7fccc1631640 1 --2- 172.21.15.44:0/1143517793 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fcc9c06c8f0 0x7fcc9c06edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fccb4036000 tx=0x7fccb4008030 comp rx=0 tx=0).stop 2024-04-22T06:31:13.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.423+0000 7fccc1631640 1 -- 172.21.15.44:0/1143517793 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccbc155340 msgr2=0x7fccbc0c9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:13.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.423+0000 7fccc1631640 1 --2- 172.21.15.44:0/1143517793 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccbc155340 0x7fccbc0c9c90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcca400c3f0 tx=0x7fcca400c8b0 comp rx=0 tx=0).stop 2024-04-22T06:31:13.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.423+0000 7fccc1631640 1 -- 172.21.15.44:0/1143517793 shutdown_connections 2024-04-22T06:31:13.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.423+0000 7fccc1631640 1 --2- 172.21.15.44:0/1143517793 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fcc9c06c8f0 0x7fcc9c06edb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:13.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.423+0000 7fccc1631640 1 --2- 172.21.15.44:0/1143517793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccbc15e610 0x7fccbc0ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:13.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.423+0000 7fccc1631640 1 --2- 172.21.15.44:0/1143517793 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccbc155340 0x7fccbc0c9c90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:13.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.423+0000 7fccc1631640 1 --2- 172.21.15.44:0/1143517793 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fccbc0cc310 0x7fccbc153e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:13.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.423+0000 7fccc1631640 1 -- 172.21.15.44:0/1143517793 >> 172.21.15.44:0/1143517793 conn(0x7fccbc06c560 msgr2=0x7fccbc14a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:13.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.424+0000 7fccc1631640 1 -- 172.21.15.44:0/1143517793 shutdown_connections 2024-04-22T06:31:13.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.424+0000 7fccc1631640 1 -- 172.21.15.44:0/1143517793 wait complete. 2024-04-22T06:31:13.433 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T06:31:13.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.535+0000 7fa41bd3d640 1 Processor -- start 2024-04-22T06:31:13.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.535+0000 7fa41bd3d640 1 -- start start 2024-04-22T06:31:13.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.536+0000 7fa41bd3d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa4141552a0 0x7fa41415bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:13.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.536+0000 7fa41bd3d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fa4140cc4e0 0x7fa4140ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:13.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.536+0000 7fa41bd3d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa41406c6e0 con 0x7fa414155680 2024-04-22T06:31:13.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.536+0000 7fa41bd3d640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fa4140cb990 con 0x7fa4140cc4e0 2024-04-22T06:31:13.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.536+0000 7fa41bd3d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fa41406c9c0 con 0x7fa4141552a0 2024-04-22T06:31:13.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.536+0000 7fa41a2da640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fa4140cc4e0 0x7fa4140ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:13.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.536+0000 7fa41a2da640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fa4140cc4e0 0x7fa4140ccfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59598/0 (socket says 172.21.15.44:59598) 2024-04-22T06:31:13.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.536+0000 7fa41a2da640 1 -- 172.21.15.44:0/1950156167 learned_addr learned my addr 172.21.15.44:0/1950156167 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:13.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.536+0000 7fa419ad9640 1 --2- 172.21.15.44:0/1950156167 >> v2:172.21.15.70:3300/0 conn(0x7fa4141552a0 0x7fa41415bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:13.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.536+0000 7fa41a2da640 1 -- 172.21.15.44:0/1950156167 >> v2:172.21.15.70:3300/0 conn(0x7fa4141552a0 msgr2=0x7fa41415bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:13.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.536+0000 7fa41a2da640 1 --2- 172.21.15.44:0/1950156167 >> v2:172.21.15.70:3300/0 conn(0x7fa4141552a0 0x7fa41415bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:13.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.536+0000 7fa41a2da640 1 -- 172.21.15.44:0/1950156167 >> v1:172.21.15.70:6789/0 conn(0x7fa414155680 legacy=0x7fa4140cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:13.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.536+0000 7fa41a2da640 1 -- 172.21.15.44:0/1950156167 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa41406cca0 con 0x7fa4140cc4e0 2024-04-22T06:31:13.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.537+0000 7fa41a2da640 1 --2- 172.21.15.44:0/1950156167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4140cc4e0 0x7fa4140ccfc0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa414073e80 tx=0x7fa40c02c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c0034dfb31e4720 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:13.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.537+0000 7fa418ad7640 1 -- 172.21.15.44:0/1950156167 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c02cbb0 con 0x7fa4140cc4e0 2024-04-22T06:31:13.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.537+0000 7fa418ad7640 1 -- 172.21.15.44:0/1950156167 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa40c02cd50 con 0x7fa4140cc4e0 2024-04-22T06:31:13.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.537+0000 7fa418ad7640 1 -- 172.21.15.44:0/1950156167 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c034610 con 0x7fa4140cc4e0 2024-04-22T06:31:13.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.537+0000 7fa41bd3d640 1 -- 172.21.15.44:0/1950156167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4140cc4e0 msgr2=0x7fa4140ccfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:13.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.537+0000 7fa41bd3d640 1 --2- 172.21.15.44:0/1950156167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4140cc4e0 0x7fa4140ccfc0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa414073e80 tx=0x7fa40c02c010 comp rx=0 tx=0).stop 2024-04-22T06:31:13.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.537+0000 7fa41bd3d640 1 -- 172.21.15.44:0/1950156167 shutdown_connections 2024-04-22T06:31:13.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.538+0000 7fa41bd3d640 1 --2- 172.21.15.44:0/1950156167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4140cc4e0 0x7fa4140ccfc0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:13.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.538+0000 7fa41bd3d640 1 --2- 172.21.15.44:0/1950156167 >> v2:172.21.15.70:3300/0 conn(0x7fa4141552a0 0x7fa41415bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:13.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.538+0000 7fa41bd3d640 1 -- 172.21.15.44:0/1950156167 >> 172.21.15.44:0/1950156167 conn(0x7fa414075820 msgr2=0x7fa414073000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:13.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.538+0000 7fa41bd3d640 1 -- 172.21.15.44:0/1950156167 shutdown_connections 2024-04-22T06:31:13.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.538+0000 7fa41bd3d640 1 -- 172.21.15.44:0/1950156167 wait complete. 2024-04-22T06:31:13.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.539+0000 7fa41bd3d640 1 Processor -- start 2024-04-22T06:31:13.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.539+0000 7fa41bd3d640 1 -- start start 2024-04-22T06:31:13.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.539+0000 7fa41bd3d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4140cc4e0 0x7fa4141e7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:13.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.539+0000 7fa41bd3d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4141552a0 0x7fa4141f3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:13.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.539+0000 7fa41bd3d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4141e4440 0x7fa4141f42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:13.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.539+0000 7fa41bd3d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa41406d540 con 0x7fa4141e4440 2024-04-22T06:31:13.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.539+0000 7fa41bd3d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa41406d260 con 0x7fa4140cc4e0 2024-04-22T06:31:13.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.539+0000 7fa41bd3d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa4140b6690 con 0x7fa4141552a0 2024-04-22T06:31:13.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.540+0000 7fa41a2da640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4141e4440 0x7fa4141f42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:13.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.540+0000 7fa41a2da640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4141e4440 0x7fa4141f42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59614/0 (socket says 172.21.15.44:59614) 2024-04-22T06:31:13.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.540+0000 7fa41a2da640 1 -- 172.21.15.44:0/616805735 learned_addr learned my addr 172.21.15.44:0/616805735 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:13.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.540+0000 7fa419ad9640 1 --2- 172.21.15.44:0/616805735 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4140cc4e0 0x7fa4141e7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:13.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.540+0000 7fa4192d8640 1 --2- 172.21.15.44:0/616805735 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4141552a0 0x7fa4141f3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:13.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.540+0000 7fa41a2da640 1 -- 172.21.15.44:0/616805735 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4141552a0 msgr2=0x7fa4141f3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:13.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.540+0000 7fa41a2da640 1 --2- 172.21.15.44:0/616805735 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4141552a0 0x7fa4141f3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:13.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.540+0000 7fa41a2da640 1 -- 172.21.15.44:0/616805735 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4140cc4e0 msgr2=0x7fa4141e7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:13.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.540+0000 7fa41a2da640 1 --2- 172.21.15.44:0/616805735 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4140cc4e0 0x7fa4141e7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:13.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.540+0000 7fa41a2da640 1 -- 172.21.15.44:0/616805735 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4141f4900 con 0x7fa4141e4440 2024-04-22T06:31:13.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.540+0000 7fa41a2da640 1 --2- 172.21.15.44:0/616805735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4141e4440 0x7fa4141f42e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa40c02c620 tx=0x7fa40c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:13.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.540+0000 7fa40a7fc640 1 -- 172.21.15.44:0/616805735 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c043020 con 0x7fa4141e4440 2024-04-22T06:31:13.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.540+0000 7fa40a7fc640 1 -- 172.21.15.44:0/616805735 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa40c03e450 con 0x7fa4141e4440 2024-04-22T06:31:13.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.540+0000 7fa41bd3d640 1 -- 172.21.15.44:0/616805735 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4141e4b20 con 0x7fa4141e4440 2024-04-22T06:31:13.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.540+0000 7fa40a7fc640 1 -- 172.21.15.44:0/616805735 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c009510 con 0x7fa4141e4440 2024-04-22T06:31:13.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.540+0000 7fa41bd3d640 1 -- 172.21.15.44:0/616805735 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4141e4dc0 con 0x7fa4141e4440 2024-04-22T06:31:13.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.541+0000 7fa41bd3d640 1 -- 172.21.15.44:0/616805735 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa414068cc0 con 0x7fa4141e4440 2024-04-22T06:31:13.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.542+0000 7fa40a7fc640 1 -- 172.21.15.44:0/616805735 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa40c046020 con 0x7fa4141e4440 2024-04-22T06:31:13.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.543+0000 7fa40a7fc640 1 --2- 172.21.15.44:0/616805735 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fa3f806c8f0 0x7fa3f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:13.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.543+0000 7fa40a7fc640 1 -- 172.21.15.44:0/616805735 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa40c0ae910 con 0x7fa4141e4440 2024-04-22T06:31:13.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.543+0000 7fa419ad9640 1 --2- 172.21.15.44:0/616805735 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fa3f806c8f0 0x7fa3f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:13.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.543+0000 7fa419ad9640 1 --2- 172.21.15.44:0/616805735 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fa3f806c8f0 0x7fa3f806edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa410009b40 tx=0x7fa410009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:13.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.546+0000 7fa40a7fc640 1 -- 172.21.15.44:0/616805735 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa40c04d030 con 0x7fa4141e4440 2024-04-22T06:31:13.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.734+0000 7fa41bd3d640 1 -- 172.21.15.44:0/616805735 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4141f3840 con 0x7fa4141e4440 2024-04-22T06:31:13.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.735+0000 7fa40a7fc640 1 -- 172.21.15.44:0/616805735 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18466 (secure 0 0 0) 0x7fa40c07cb90 con 0x7fa4141e4440 2024-04-22T06:31:13.734 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:31:13.734 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"fsid":"9b0a34ca-403d-4513-8a66-d3749ba559aa","created":"2024-04-22T06:30:42.334581+0000","modified":"2024-04-22T06:31:11.406256+0000","last_up_change":"2024-04-22T06:30:52.942257+0000","last_in_change":"2024-04-22T06:30:44.467727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T06:30:54.560256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T06:31:04.612387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T06:31:05.312124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-22T06:31:09.025550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b4a6692-53c9-472d-b2f3-1dd29a6679ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6817","nonce":2220268493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6819","nonce":2220268493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6823","nonce":2220268493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6821","nonce":2220268493}]},"public_addr":"172.21.15.44:6817/2220268493","cluster_addr":"172.21.15.44:6819/2220268493","heartbeat_back_addr":"172.21.15.44:6823/2220268493","heartbeat_front_addr":"172.21.15.44:6821/2220268493","state":["exists","up"]},{"osd":1,"uuid":"70a6370a-7c0a-4d64-a0e4-b2e726b4f291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6809","nonce":1278678977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6811","nonce":1278678977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6815","nonce":1278678977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6813","nonce":1278678977}]},"public_addr":"172.21.15.44:6809/1278678977","cluster_addr":"172.21.15.44:6811/1278678977","heartbeat_back_addr":"172.21.15.44:6815/1278678977","heartbeat_front_addr":"172.21.15.44:6813/1278678977","state":["exists","up"]},{"osd":2,"uuid":"f2e51a62-ca91-470d-bd5a-5710c61deee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6801","nonce":2544087903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6803","nonce":2544087903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6807","nonce":2544087903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6805","nonce":2544087903}]},"public_addr":"172.21.15.44:6801/2544087903","cluster_addr":"172.21.15.44:6803/2544087903","heartbeat_back_addr":"172.21.15.44:6807/2544087903","heartbeat_front_addr":"172.21.15.44:6805/2544087903","state":["exists","up"]},{"osd":3,"uuid":"bc514203-9af9-4f1f-b18e-765bdd5de6e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6825","nonce":1692928109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6827","nonce":1692928109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6831","nonce":1692928109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6829","nonce":1692928109}]},"public_addr":"172.21.15.44:6825/1692928109","cluster_addr":"172.21.15.44:6827/1692928109","heartbeat_back_addr":"172.21.15.44:6831/1692928109","heartbeat_front_addr":"172.21.15.44:6829/1692928109","state":["exists","up"]},{"osd":4,"uuid":"357c439a-6e08-4c19-ac18-b68e6c890497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6820","nonce":3802577427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6824","nonce":3802577427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6831","nonce":3802577427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6828","nonce":3802577427}]},"public_addr":"172.21.15.70:6820/3802577427","cluster_addr":"172.21.15.70:6824/3802577427","heartbeat_back_addr":"172.21.15.70:6831/3802577427","heartbeat_front_addr":"172.21.15.70:6828/3802577427","state":["exists","up"]},{"osd":5,"uuid":"952fb88f-d56e-4305-bfa8-d32674332dfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6801","nonce":664757345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6803","nonce":664757345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6807","nonce":664757345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6805","nonce":664757345}]},"public_addr":"172.21.15.70:6801/664757345","cluster_addr":"172.21.15.70:6803/664757345","heartbeat_back_addr":"172.21.15.70:6807/664757345","heartbeat_front_addr":"172.21.15.70:6805/664757345","state":["exists","up"]},{"osd":6,"uuid":"74399b3d-fac2-45ab-9172-2cd95b02450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6817","nonce":1623253482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6821","nonce":1623253482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6827","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6829","nonce":1623253482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6825","nonce":1623253482}]},"public_addr":"172.21.15.70:6817/1623253482","cluster_addr":"172.21.15.70:6821/1623253482","heartbeat_back_addr":"172.21.15.70:6829/1623253482","heartbeat_front_addr":"172.21.15.70:6825/1623253482","state":["exists","up"]},{"osd":7,"uuid":"8f35508d-6fcc-43ff-8ee7-0498a7ab3b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6809","nonce":1551180300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6811","nonce":1551180300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6815","nonce":1551180300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6813","nonce":1551180300}]},"public_addr":"172.21.15.70:6809/1551180300","cluster_addr":"172.21.15.70:6811/1551180300","heartbeat_back_addr":"172.21.15.70:6815/1551180300","heartbeat_front_addr":"172.21.15.70:6813/1551180300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.612502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.653756+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.521169+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.649955+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.640231+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.686638+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.642199+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.700360+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T06:31:13.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.737+0000 7fa41bd3d640 1 -- 172.21.15.44:0/616805735 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fa3f806c8f0 msgr2=0x7fa3f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:13.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.737+0000 7fa41bd3d640 1 --2- 172.21.15.44:0/616805735 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fa3f806c8f0 0x7fa3f806edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa410009b40 tx=0x7fa410009190 comp rx=0 tx=0).stop 2024-04-22T06:31:13.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.737+0000 7fa41bd3d640 1 -- 172.21.15.44:0/616805735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4141e4440 msgr2=0x7fa4141f42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:13.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.737+0000 7fa41bd3d640 1 --2- 172.21.15.44:0/616805735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4141e4440 0x7fa4141f42e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa40c02c620 tx=0x7fa40c004a40 comp rx=0 tx=0).stop 2024-04-22T06:31:13.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.737+0000 7fa41bd3d640 1 -- 172.21.15.44:0/616805735 shutdown_connections 2024-04-22T06:31:13.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.737+0000 7fa41bd3d640 1 --2- 172.21.15.44:0/616805735 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fa3f806c8f0 0x7fa3f806edb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:13.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.737+0000 7fa41bd3d640 1 --2- 172.21.15.44:0/616805735 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4141552a0 0x7fa4141f3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:13.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.737+0000 7fa41bd3d640 1 --2- 172.21.15.44:0/616805735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4141e4440 0x7fa4141f42e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:13.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.737+0000 7fa41bd3d640 1 --2- 172.21.15.44:0/616805735 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4140cc4e0 0x7fa4141e7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:13.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.737+0000 7fa41bd3d640 1 -- 172.21.15.44:0/616805735 >> 172.21.15.44:0/616805735 conn(0x7fa414075820 msgr2=0x7fa414072a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:13.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.737+0000 7fa41bd3d640 1 -- 172.21.15.44:0/616805735 shutdown_connections 2024-04-22T06:31:13.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:13.737+0000 7fa41bd3d640 1 -- 172.21.15.44:0/616805735 wait complete. 2024-04-22T06:31:14.350 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-22T06:31:06.356312+0000', 'modified': '2024-04-22T06:31:12.025959+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4308}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4308': {'gid': 4308, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.70:6839/2915150040', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6837', 'nonce': 2915150040}, {'type': 'v1', 'addr': '172.21.15.70:6839', 'nonce': 2915150040}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4308, 'qdb_cluster': [4308]} 2024-04-22T06:31:14.351 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-22T06:31:14.351 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-22T06:31:14.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.466+0000 7faf8913b640 1 Processor -- start 2024-04-22T06:31:14.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.466+0000 7faf8913b640 1 -- start start 2024-04-22T06:31:14.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.466+0000 7faf8913b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7faf841552e0 0x7faf840cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:14.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.466+0000 7faf8913b640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faf8406d470 0x7faf840d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:14.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.466+0000 7faf8913b640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf8415ccf0 con 0x7faf84154f00 2024-04-22T06:31:14.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.466+0000 7faf8913b640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7faf840c6740 con 0x7faf841552e0 2024-04-22T06:31:14.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.466+0000 7faf8913b640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7faf840c5b30 con 0x7faf8406d470 2024-04-22T06:31:14.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.467+0000 7faf8259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7faf841552e0 0x7faf840cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:14.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.467+0000 7faf8359e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faf8406d470 0x7faf840d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:14.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.467+0000 7faf82d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7faf84154f00 0x7faf8415c5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.44:43382/0 (socket says 172.21.15.44:43382) 2024-04-22T06:31:14.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.467+0000 7faf82d9d640 1 -- 172.21.15.44:0/3211522392 learned_addr learned my addr 172.21.15.44:0/3211522392 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:14.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.467+0000 7faf8259c640 1 -- 172.21.15.44:0/3211522392 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faf8406d470 msgr2=0x7faf840d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:14.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.467+0000 7faf8259c640 1 --2- 172.21.15.44:0/3211522392 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faf8406d470 0x7faf840d0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:14.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.467+0000 7faf8259c640 1 -- 172.21.15.44:0/3211522392 >> v1:172.21.15.70:6789/0 conn(0x7faf84154f00 legacy=0x7faf8415c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:14.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.467+0000 7faf8259c640 1 -- 172.21.15.44:0/3211522392 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf840c5e10 con 0x7faf841552e0 2024-04-22T06:31:14.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.467+0000 7faf81d9b640 1 -- 172.21.15.44:0/3211522392 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1278813593 0 0) 0x7faf8415ccf0 con 0x7faf84154f00 2024-04-22T06:31:14.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.468+0000 7faf8259c640 1 --2- 172.21.15.44:0/3211522392 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf841552e0 0x7faf840cb430 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7faf780097a0 tx=0x7faf7802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ecda83ec48b50df server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:14.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.468+0000 7faf81d9b640 1 -- 172.21.15.44:0/3211522392 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf78035020 con 0x7faf841552e0 2024-04-22T06:31:14.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.468+0000 7faf81d9b640 1 -- 172.21.15.44:0/3211522392 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf78031830 con 0x7faf841552e0 2024-04-22T06:31:14.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.468+0000 7faf81d9b640 1 -- 172.21.15.44:0/3211522392 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf78031c00 con 0x7faf841552e0 2024-04-22T06:31:14.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.469+0000 7faf8913b640 1 -- 172.21.15.44:0/3211522392 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf841552e0 msgr2=0x7faf840cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:14.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.469+0000 7faf8913b640 1 --2- 172.21.15.44:0/3211522392 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf841552e0 0x7faf840cb430 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7faf780097a0 tx=0x7faf7802b8d0 comp rx=0 tx=0).stop 2024-04-22T06:31:14.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.469+0000 7faf8913b640 1 -- 172.21.15.44:0/3211522392 shutdown_connections 2024-04-22T06:31:14.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.469+0000 7faf8913b640 1 --2- 172.21.15.44:0/3211522392 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faf8406d470 0x7faf840d0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:14.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.469+0000 7faf8913b640 1 --2- 172.21.15.44:0/3211522392 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf841552e0 0x7faf840cb430 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:14.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.469+0000 7faf8913b640 1 -- 172.21.15.44:0/3211522392 >> 172.21.15.44:0/3211522392 conn(0x7faf840733c0 msgr2=0x7faf840737d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:14.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.469+0000 7faf8913b640 1 -- 172.21.15.44:0/3211522392 shutdown_connections 2024-04-22T06:31:14.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.469+0000 7faf8913b640 1 -- 172.21.15.44:0/3211522392 wait complete. 2024-04-22T06:31:14.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.470+0000 7faf8913b640 1 Processor -- start 2024-04-22T06:31:14.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.471+0000 7faf8913b640 1 -- start start 2024-04-22T06:31:14.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.471+0000 7faf8913b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf8406d470 0x7faf841f1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:14.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.471+0000 7faf8913b640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faf84154f00 0x7faf840b5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:14.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.471+0000 7faf8913b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf841552e0 0x7faf8415a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:14.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.471+0000 7faf8913b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7faf840c6bd0 con 0x7faf841552e0 2024-04-22T06:31:14.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.471+0000 7faf8913b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7faf840c63d0 con 0x7faf8406d470 2024-04-22T06:31:14.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.471+0000 7faf8913b640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7faf840c6d50 con 0x7faf84154f00 2024-04-22T06:31:14.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.471+0000 7faf8359e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf841552e0 0x7faf8415a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:14.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.471+0000 7faf8359e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf841552e0 0x7faf8415a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59628/0 (socket says 172.21.15.44:59628) 2024-04-22T06:31:14.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.471+0000 7faf8359e640 1 -- 172.21.15.44:0/2646801849 learned_addr learned my addr 172.21.15.44:0/2646801849 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:14.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.471+0000 7faf82d9d640 1 --2- 172.21.15.44:0/2646801849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf8406d470 0x7faf841f1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:14.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.472+0000 7faf8259c640 1 --2- 172.21.15.44:0/2646801849 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faf84154f00 0x7faf840b5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:14.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.472+0000 7faf8359e640 1 -- 172.21.15.44:0/2646801849 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faf84154f00 msgr2=0x7faf840b5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:14.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.472+0000 7faf8359e640 1 --2- 172.21.15.44:0/2646801849 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faf84154f00 0x7faf840b5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:14.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.472+0000 7faf8359e640 1 -- 172.21.15.44:0/2646801849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf8406d470 msgr2=0x7faf841f1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:14.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.472+0000 7faf8359e640 1 --2- 172.21.15.44:0/2646801849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf8406d470 0x7faf841f1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:14.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.472+0000 7faf8359e640 1 -- 172.21.15.44:0/2646801849 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf84159340 con 0x7faf841552e0 2024-04-22T06:31:14.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.472+0000 7faf8359e640 1 --2- 172.21.15.44:0/2646801849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf841552e0 0x7faf8415a660 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7faf7c009dd0 tx=0x7faf7c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:14.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.472+0000 7faf6b7fe640 1 -- 172.21.15.44:0/2646801849 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c018020 con 0x7faf841552e0 2024-04-22T06:31:14.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.472+0000 7faf6b7fe640 1 -- 172.21.15.44:0/2646801849 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf7c014480 con 0x7faf841552e0 2024-04-22T06:31:14.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.472+0000 7faf8913b640 1 -- 172.21.15.44:0/2646801849 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf84159600 con 0x7faf841552e0 2024-04-22T06:31:14.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.472+0000 7faf6b7fe640 1 -- 172.21.15.44:0/2646801849 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c014870 con 0x7faf841552e0 2024-04-22T06:31:14.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.472+0000 7faf8913b640 1 -- 172.21.15.44:0/2646801849 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf84159e60 con 0x7faf841552e0 2024-04-22T06:31:14.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.474+0000 7faf6b7fe640 1 -- 172.21.15.44:0/2646801849 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faf7c005d10 con 0x7faf841552e0 2024-04-22T06:31:14.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.474+0000 7faf6b7fe640 1 --2- 172.21.15.44:0/2646801849 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7faf7006c8f0 0x7faf7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:14.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.475+0000 7faf6b7fe640 1 -- 172.21.15.44:0/2646801849 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7faf7c08c910 con 0x7faf841552e0 2024-04-22T06:31:14.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.475+0000 7faf82d9d640 1 --2- 172.21.15.44:0/2646801849 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7faf7006c8f0 0x7faf7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:14.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.475+0000 7faf82d9d640 1 --2- 172.21.15.44:0/2646801849 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7faf7006c8f0 0x7faf7006edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faf74000f60 tx=0x7faf74007d50 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:14.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.476+0000 7faf8be05740 1 Processor -- start 2024-04-22T06:31:14.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.476+0000 7faf8be05740 1 -- start start 2024-04-22T06:31:14.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.476+0000 7faf8be05740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e25af93e40 0x55e25af9cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:14.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.476+0000 7faf8be05740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e25af54ec0 0x55e25af9d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:14.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.476+0000 7faf8be05740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e25af50ca0 0x55e25af78ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:14.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.476+0000 7faf8be05740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e25aeb7b10 con 0x55e25af93e40 2024-04-22T06:31:14.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.476+0000 7faf8be05740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55e25ae55240 con 0x55e25af54ec0 2024-04-22T06:31:14.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.476+0000 7faf8be05740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55e25aeb3ad0 con 0x55e25af50ca0 2024-04-22T06:31:14.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.477+0000 7faf82d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e25af93e40 0x55e25af9cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:14.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.477+0000 7faf82d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e25af93e40 0x55e25af9cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59636/0 (socket says 172.21.15.44:59636) 2024-04-22T06:31:14.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.477+0000 7faf82d9d640 1 -- 172.21.15.44:0/774272144 learned_addr learned my addr 172.21.15.44:0/774272144 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:14.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.477+0000 7faf8259c640 1 --2- 172.21.15.44:0/774272144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e25af54ec0 0x55e25af9d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:14.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.477+0000 7faf8359e640 1 --2- 172.21.15.44:0/774272144 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e25af50ca0 0x55e25af78ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:14.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.477+0000 7faf82d9d640 1 -- 172.21.15.44:0/774272144 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e25af50ca0 msgr2=0x55e25af78ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:14.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.477+0000 7faf82d9d640 1 --2- 172.21.15.44:0/774272144 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e25af50ca0 0x55e25af78ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:14.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.477+0000 7faf82d9d640 1 -- 172.21.15.44:0/774272144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e25af54ec0 msgr2=0x55e25af9d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:14.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.477+0000 7faf82d9d640 1 --2- 172.21.15.44:0/774272144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e25af54ec0 0x55e25af9d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:14.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.477+0000 7faf82d9d640 1 -- 172.21.15.44:0/774272144 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e25abca2d0 con 0x55e25af93e40 2024-04-22T06:31:14.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.477+0000 7faf8259c640 1 --2- 172.21.15.44:0/774272144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e25af54ec0 0x55e25af9d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:31:14.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.477+0000 7faf82d9d640 1 --2- 172.21.15.44:0/774272144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e25af93e40 0x55e25af9cf60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7faf74007aa0 tx=0x7faf74013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd5ab0f544c7e8f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:14.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.477+0000 7faf68ff9640 1 -- 172.21.15.44:0/774272144 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf74026070 con 0x55e25af93e40 2024-04-22T06:31:14.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.477+0000 7faf68ff9640 1 -- 172.21.15.44:0/774272144 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf74013b80 con 0x55e25af93e40 2024-04-22T06:31:14.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.477+0000 7faf68ff9640 1 -- 172.21.15.44:0/774272144 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf74012350 con 0x55e25af93e40 2024-04-22T06:31:14.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.478+0000 7faf8be05740 1 -- 172.21.15.44:0/774272144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e25af93e40 msgr2=0x55e25af9cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:14.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.478+0000 7faf8be05740 1 --2- 172.21.15.44:0/774272144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e25af93e40 0x55e25af9cf60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7faf74007aa0 tx=0x7faf74013000 comp rx=0 tx=0).stop 2024-04-22T06:31:14.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.478+0000 7faf8be05740 1 -- 172.21.15.44:0/774272144 shutdown_connections 2024-04-22T06:31:14.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.478+0000 7faf8be05740 1 --2- 172.21.15.44:0/774272144 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e25af50ca0 0x55e25af78ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:14.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.478+0000 7faf8be05740 1 --2- 172.21.15.44:0/774272144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e25af54ec0 0x55e25af9d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:14.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.478+0000 7faf8be05740 1 --2- 172.21.15.44:0/774272144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e25af93e40 0x55e25af9cf60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:14.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.478+0000 7faf8be05740 1 -- 172.21.15.44:0/774272144 >> 172.21.15.44:0/774272144 conn(0x55e25af4f990 msgr2=0x55e25af52290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:14.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.478+0000 7faf8be05740 1 -- 172.21.15.44:0/774272144 shutdown_connections 2024-04-22T06:31:14.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.478+0000 7faf8be05740 1 -- 172.21.15.44:0/774272144 wait complete. 2024-04-22T06:31:14.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.479+0000 7faf8be05740 1 -- start start 2024-04-22T06:31:14.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.479+0000 7faf8be05740 1 Processor -- start 2024-04-22T06:31:14.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.479+0000 7faf8be05740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e25af54ec0 0x55e25af9bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:14.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.479+0000 7faf8be05740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e25af93e40 0x55e25af4cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:14.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.479+0000 7faf8be05740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e25af95ea0 0x55e25afa7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:14.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.479+0000 7faf8be05740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e25af53250 con 0x55e25af95ea0 2024-04-22T06:31:14.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.479+0000 7faf8be05740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55e25ae40190 con 0x55e25af93e40 2024-04-22T06:31:14.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.479+0000 7faf8be05740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55e25af53530 con 0x55e25af54ec0 2024-04-22T06:31:14.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.479+0000 7faf8be05740 10 client.0 _ll_register_callbacks cb 0x55e25ae3b8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T06:31:14.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.479+0000 7faf8359e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e25af95ea0 0x55e25afa7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:14.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.479+0000 7faf8359e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e25af95ea0 0x55e25afa7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59640/0 (socket says 172.21.15.44:59640) 2024-04-22T06:31:14.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.479+0000 7faf8359e640 1 -- 172.21.15.44:0/596368973 learned_addr learned my addr 172.21.15.44:0/596368973 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:14.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.480+0000 7faf82d9d640 1 --2- 172.21.15.44:0/596368973 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e25af54ec0 0x55e25af9bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:14.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.480+0000 7faf8259c640 1 --2- 172.21.15.44:0/596368973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e25af93e40 0x55e25af4cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:14.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.480+0000 7faf8359e640 1 -- 172.21.15.44:0/596368973 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e25af54ec0 msgr2=0x55e25af9bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:14.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.480+0000 7faf8359e640 1 --2- 172.21.15.44:0/596368973 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e25af54ec0 0x55e25af9bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:14.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.480+0000 7faf8359e640 1 -- 172.21.15.44:0/596368973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e25af93e40 msgr2=0x55e25af4cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:14.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.480+0000 7faf8359e640 1 --2- 172.21.15.44:0/596368973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e25af93e40 0x55e25af4cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:14.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.480+0000 7faf8359e640 1 -- 172.21.15.44:0/596368973 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e25ac946b0 con 0x55e25af95ea0 2024-04-22T06:31:14.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.480+0000 7faf8259c640 1 --2- 172.21.15.44:0/596368973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e25af93e40 0x55e25af4cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:31:14.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.480+0000 7faf8359e640 1 --2- 172.21.15.44:0/596368973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e25af95ea0 0x55e25afa7ca0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7faf7c00c990 tx=0x7faf7c062ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec8d6e04c06c577 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:14.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.480+0000 7faf61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T06:31:14.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.480+0000 7faf61ffb640 1 -- 172.21.15.44:0/596368973 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c05e150 con 0x55e25af95ea0 2024-04-22T06:31:14.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.480+0000 7faf61ffb640 1 -- 172.21.15.44:0/596368973 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7faf7c05e2f0 con 0x55e25af95ea0 2024-04-22T06:31:14.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.480+0000 7faf8be05740 1 -- 172.21.15.44:0/596368973 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf74013b80 con 0x55e25af95ea0 2024-04-22T06:31:14.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.480+0000 7faf61ffb640 1 -- 172.21.15.44:0/596368973 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7faf7c059770 con 0x55e25af95ea0 2024-04-22T06:31:14.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.480+0000 7faf8be05740 10 client.4443 fetch_fsmap learned FSMap version 6 2024-04-22T06:31:14.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.480+0000 7faf8be05740 1 -- 172.21.15.44:0/596368973 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x55e25af4c410 con 0x55e25af95ea0 2024-04-22T06:31:14.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.480+0000 7faf61ffb640 1 -- 172.21.15.44:0/596368973 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf74013b80 con 0x55e25af95ea0 2024-04-22T06:31:14.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.480+0000 7faf61ffb640 1 -- 172.21.15.44:0/596368973 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 6) ==== 4712+0+0 (secure 0 0 0) 0x7faf7c059910 con 0x55e25af95ea0 2024-04-22T06:31:14.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.481+0000 7faf8be05740 10 client.4443 fetch_fsmap finished waiting for FSMap version 6 2024-04-22T06:31:14.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.481+0000 7faf8be05740 10 client.4443 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-22T06:31:14.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.481+0000 7faf8be05740 20 client.4443 populate_metadata read hostname 'smithi044' 2024-04-22T06:31:14.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.481+0000 7faf8be05740 1 --2- 172.21.15.44:0/596368973 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x55e25afa8830 0x55e25afb3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:14.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.481+0000 7faf8be05740 4 client.4443 mds_command: new command op to 4308 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T06:31:14.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.481+0000 7faf8be05740 1 -- 172.21.15.44:0/596368973 --> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e25aec5530 con 0x55e25afa8830 2024-04-22T06:31:14.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.481+0000 7faf82d9d640 1 --2- 172.21.15.44:0/596368973 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x55e25afa8830 0x55e25afb3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:14.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.481+0000 7faf61ffb640 10 client.4443 ms_handle_connect on v2:172.21.15.70:6837/2915150040 2024-04-22T06:31:14.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:14.481+0000 7faf82d9d640 1 --2- 172.21.15.44:0/596368973 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x55e25afa8830 0x55e25afb3950 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5461fd4408ee5ecf server_cookie=7c9546e2666e2d02 in_seq=0 out_seq=0 2024-04-22T06:31:17.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.029+0000 7faf61ffb640 1 -- 172.21.15.44:0/596368973 <== mds.0 v2:172.21.15.70:6837/2915150040 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e25aec5530 con 0x55e25afa8830 2024-04-22T06:31:17.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.029+0000 7faf61ffb640 10 client.4443 handle_command_reply: tid=0 2024-04-22T06:31:17.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.029+0000 7faf8be05740 1 client.4443 shutdown 2024-04-22T06:31:17.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.029+0000 7faf8be05740 1 -- 172.21.15.44:0/596368973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e25af95ea0 msgr2=0x55e25afa7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:17.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.030+0000 7faf8be05740 1 --2- 172.21.15.44:0/596368973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e25af95ea0 0x55e25afa7ca0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7faf7c00c990 tx=0x7faf7c062ee0 comp rx=0 tx=0).stop 2024-04-22T06:31:17.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.030+0000 7faf8be05740 1 -- 172.21.15.44:0/596368973 shutdown_connections 2024-04-22T06:31:17.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.030+0000 7faf8be05740 1 --2- 172.21.15.44:0/596368973 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x55e25afa8830 0x55e25afb3950 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:17.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.030+0000 7faf8be05740 1 --2- 172.21.15.44:0/596368973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e25af95ea0 0x55e25afa7ca0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:17.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.030+0000 7faf8be05740 1 --2- 172.21.15.44:0/596368973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e25af93e40 0x55e25af4cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:17.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.030+0000 7faf8be05740 1 --2- 172.21.15.44:0/596368973 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e25af54ec0 0x55e25af9bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:17.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.030+0000 7faf61ffb640 0 client.4443 ms_handle_reset on v2:172.21.15.70:6837/2915150040 2024-04-22T06:31:17.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.030+0000 7faf8be05740 1 -- 172.21.15.44:0/596368973 >> 172.21.15.44:0/596368973 conn(0x55e25af50ca0 msgr2=0x55e25af92bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:17.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.030+0000 7faf8be05740 1 -- 172.21.15.44:0/596368973 shutdown_connections 2024-04-22T06:31:17.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.030+0000 7faf8be05740 1 -- 172.21.15.44:0/596368973 wait complete. 2024-04-22T06:31:17.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.030+0000 7faf8be05740 20 client.4443 trim_cache size 0 max 16384 2024-04-22T06:31:17.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.050+0000 7faf8be05740 1 Processor -- start 2024-04-22T06:31:17.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.050+0000 7faf8be05740 1 -- start start 2024-04-22T06:31:17.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.051+0000 7faf8be05740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e25af54ec0 0x55e25afae070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:17.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.051+0000 7faf8be05740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e25af93e40 0x55e25af78dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:17.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.051+0000 7faf8be05740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e25af95ea0 0x55e25af79300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:17.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.051+0000 7faf8be05740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7faf841fd6e0 con 0x55e25af95ea0 2024-04-22T06:31:17.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.051+0000 7faf8be05740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55e25af9c510 con 0x55e25af93e40 2024-04-22T06:31:17.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.051+0000 7faf8be05740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55e25af9a450 con 0x55e25af54ec0 2024-04-22T06:31:17.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.051+0000 7faf8359e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e25af95ea0 0x55e25af79300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:17.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.051+0000 7faf8359e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e25af95ea0 0x55e25af79300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59642/0 (socket says 172.21.15.44:59642) 2024-04-22T06:31:17.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.051+0000 7faf8359e640 1 -- 172.21.15.44:0/2522744698 learned_addr learned my addr 172.21.15.44:0/2522744698 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:17.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.051+0000 7faf8259c640 1 --2- 172.21.15.44:0/2522744698 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e25af93e40 0x55e25af78dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:17.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.051+0000 7faf82d9d640 1 --2- 172.21.15.44:0/2522744698 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e25af54ec0 0x55e25afae070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:17.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.051+0000 7faf8359e640 1 -- 172.21.15.44:0/2522744698 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e25af54ec0 msgr2=0x55e25afae070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:17.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.051+0000 7faf8359e640 1 --2- 172.21.15.44:0/2522744698 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e25af54ec0 0x55e25afae070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:17.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.051+0000 7faf8359e640 1 -- 172.21.15.44:0/2522744698 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e25af93e40 msgr2=0x55e25af78dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:17.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.051+0000 7faf8359e640 1 --2- 172.21.15.44:0/2522744698 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e25af93e40 0x55e25af78dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:17.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.051+0000 7faf8359e640 1 -- 172.21.15.44:0/2522744698 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e25af50720 con 0x55e25af95ea0 2024-04-22T06:31:17.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.051+0000 7faf8359e640 1 --2- 172.21.15.44:0/2522744698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e25af95ea0 0x55e25af79300 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7faf7c05a020 tx=0x7faf7c01f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7994783c1e737bf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:17.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.052+0000 7faf63fff640 1 -- 172.21.15.44:0/2522744698 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c0583c0 con 0x55e25af95ea0 2024-04-22T06:31:17.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.052+0000 7faf63fff640 1 -- 172.21.15.44:0/2522744698 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf7c059910 con 0x55e25af95ea0 2024-04-22T06:31:17.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.052+0000 7faf63fff640 1 -- 172.21.15.44:0/2522744698 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c05e350 con 0x55e25af95ea0 2024-04-22T06:31:17.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.052+0000 7faf8be05740 1 -- 172.21.15.44:0/2522744698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e25af95ea0 msgr2=0x55e25af79300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:17.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.052+0000 7faf8be05740 1 --2- 172.21.15.44:0/2522744698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e25af95ea0 0x55e25af79300 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7faf7c05a020 tx=0x7faf7c01f070 comp rx=0 tx=0).stop 2024-04-22T06:31:17.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.052+0000 7faf8be05740 1 -- 172.21.15.44:0/2522744698 shutdown_connections 2024-04-22T06:31:17.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.053+0000 7faf8be05740 1 --2- 172.21.15.44:0/2522744698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e25af95ea0 0x55e25af79300 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:17.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.053+0000 7faf8be05740 1 --2- 172.21.15.44:0/2522744698 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e25af93e40 0x55e25af78dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:17.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.053+0000 7faf8be05740 1 --2- 172.21.15.44:0/2522744698 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e25af54ec0 0x55e25afae070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:17.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.053+0000 7faf8be05740 1 -- 172.21.15.44:0/2522744698 >> 172.21.15.44:0/2522744698 conn(0x55e25af50ca0 msgr2=0x55e25afa7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:17.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.053+0000 7faf8be05740 1 -- 172.21.15.44:0/2522744698 shutdown_connections 2024-04-22T06:31:17.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.053+0000 7faf8be05740 1 -- 172.21.15.44:0/2522744698 wait complete. 2024-04-22T06:31:17.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.053+0000 7faf8be05740 1 -- start start 2024-04-22T06:31:17.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.053+0000 7faf8be05740 1 Processor -- start 2024-04-22T06:31:17.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.054+0000 7faf8be05740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e25af54ec0 0x55e25b013480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:17.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.054+0000 7faf8be05740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e25af93e40 0x55e25af4dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:17.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.054+0000 7faf8be05740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e25af95ea0 0x55e25b0092e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:17.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.054+0000 7faf8be05740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e25af8d270 con 0x55e25af93e40 2024-04-22T06:31:17.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.054+0000 7faf8be05740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55e25afa86b0 con 0x55e25af54ec0 2024-04-22T06:31:17.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.054+0000 7faf8be05740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55e25af8d920 con 0x55e25af95ea0 2024-04-22T06:31:17.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.054+0000 7faf8be05740 10 client.0 _ll_register_callbacks cb 0x7faf7c05a410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T06:31:17.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.054+0000 7faf8259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e25af93e40 0x55e25af4dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:17.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.054+0000 7faf8259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e25af93e40 0x55e25af4dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59644/0 (socket says 172.21.15.44:59644) 2024-04-22T06:31:17.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.054+0000 7faf8259c640 1 -- 172.21.15.44:0/2690912370 learned_addr learned my addr 172.21.15.44:0/2690912370 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:17.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.054+0000 7faf82d9d640 1 --2- 172.21.15.44:0/2690912370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e25af54ec0 0x55e25b013480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:17.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.054+0000 7faf8359e640 1 --2- 172.21.15.44:0/2690912370 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e25af95ea0 0x55e25b0092e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:17.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.055+0000 7faf8259c640 1 -- 172.21.15.44:0/2690912370 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e25af95ea0 msgr2=0x55e25b0092e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:17.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.055+0000 7faf8259c640 1 --2- 172.21.15.44:0/2690912370 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e25af95ea0 0x55e25b0092e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:17.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.055+0000 7faf8259c640 1 -- 172.21.15.44:0/2690912370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e25af54ec0 msgr2=0x55e25b013480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:17.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.055+0000 7faf8259c640 1 --2- 172.21.15.44:0/2690912370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e25af54ec0 0x55e25b013480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:17.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.055+0000 7faf8259c640 1 -- 172.21.15.44:0/2690912370 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e25af4e480 con 0x55e25af93e40 2024-04-22T06:31:17.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.055+0000 7faf82d9d640 1 --2- 172.21.15.44:0/2690912370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e25af54ec0 0x55e25b013480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:31:17.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.055+0000 7faf8259c640 1 --2- 172.21.15.44:0/2690912370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e25af93e40 0x55e25af4dd50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7faf78034040 tx=0x7faf7802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac849d95bb261be server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:17.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.055+0000 7faf61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T06:31:17.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.055+0000 7faf61ffb640 1 -- 172.21.15.44:0/2690912370 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7804c060 con 0x55e25af93e40 2024-04-22T06:31:17.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.055+0000 7faf61ffb640 1 -- 172.21.15.44:0/2690912370 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7faf78031a40 con 0x55e25af93e40 2024-04-22T06:31:17.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.055+0000 7faf8be05740 1 -- 172.21.15.44:0/2690912370 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf7c059910 con 0x55e25af93e40 2024-04-22T06:31:17.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.055+0000 7faf61ffb640 1 -- 172.21.15.44:0/2690912370 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7faf78052050 con 0x55e25af93e40 2024-04-22T06:31:17.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.055+0000 7faf8be05740 10 client.4449 fetch_fsmap learned FSMap version 6 2024-04-22T06:31:17.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.055+0000 7faf8be05740 1 -- 172.21.15.44:0/2690912370 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x55e25af4e760 con 0x55e25af93e40 2024-04-22T06:31:17.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.055+0000 7faf61ffb640 1 -- 172.21.15.44:0/2690912370 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf7c059910 con 0x55e25af93e40 2024-04-22T06:31:17.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.056+0000 7faf61ffb640 1 -- 172.21.15.44:0/2690912370 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 6) ==== 4712+0+0 (secure 0 0 0) 0x7faf78055020 con 0x55e25af93e40 2024-04-22T06:31:17.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.056+0000 7faf8be05740 10 client.4449 fetch_fsmap finished waiting for FSMap version 6 2024-04-22T06:31:17.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.056+0000 7faf8be05740 10 client.4449 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-22T06:31:17.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.056+0000 7faf8be05740 20 client.4449 populate_metadata read hostname 'smithi044' 2024-04-22T06:31:17.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.056+0000 7faf8be05740 1 --2- 172.21.15.44:0/2690912370 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x55e25b010010 0x55e25b01be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:17.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.056+0000 7faf8be05740 4 client.4449 mds_command: new command op to 4308 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T06:31:17.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.056+0000 7faf8be05740 1 -- 172.21.15.44:0/2690912370 --> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] -- command(tid 0: {"prefix": "status"}) -- 0x55e25aebf3a0 con 0x55e25b010010 2024-04-22T06:31:17.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.056+0000 7faf82d9d640 1 --2- 172.21.15.44:0/2690912370 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x55e25b010010 0x55e25b01be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:17.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.056+0000 7faf61ffb640 10 client.4449 ms_handle_connect on v2:172.21.15.70:6837/2915150040 2024-04-22T06:31:17.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.056+0000 7faf82d9d640 1 --2- 172.21.15.44:0/2690912370 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x55e25b010010 0x55e25b01be10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88596ee5ad9bd63e server_cookie=c9069c9c7e10b96e in_seq=0 out_seq=0 2024-04-22T06:31:17.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.057+0000 7faf61ffb640 1 -- 172.21.15.44:0/2690912370 <== mds.0 v2:172.21.15.70:6837/2915150040 1 ==== command_reply(tid 0: 0 ) ==== 8+0+318 (crc 0 0 0) 0x55e25aebf3a0 con 0x55e25b010010 2024-04-22T06:31:17.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.057+0000 7faf61ffb640 10 client.4449 handle_command_reply: tid=0 2024-04-22T06:31:17.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.057+0000 7faf8be05740 1 client.4449 shutdown 2024-04-22T06:31:17.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.057+0000 7faf8be05740 1 -- 172.21.15.44:0/2690912370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e25af93e40 msgr2=0x55e25af4dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:17.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.057+0000 7faf8be05740 1 --2- 172.21.15.44:0/2690912370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e25af93e40 0x55e25af4dd50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7faf78034040 tx=0x7faf7802e000 comp rx=0 tx=0).stop 2024-04-22T06:31:17.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.057+0000 7faf8be05740 1 -- 172.21.15.44:0/2690912370 shutdown_connections 2024-04-22T06:31:17.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.057+0000 7faf8be05740 1 --2- 172.21.15.44:0/2690912370 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e25af95ea0 0x55e25b0092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:17.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.057+0000 7faf8be05740 1 --2- 172.21.15.44:0/2690912370 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x55e25b010010 0x55e25b01be10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:17.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.057+0000 7faf8be05740 1 --2- 172.21.15.44:0/2690912370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e25af93e40 0x55e25af4dd50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:17.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.057+0000 7faf8be05740 1 --2- 172.21.15.44:0/2690912370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e25af54ec0 0x55e25b013480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:17.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.057+0000 7faf61ffb640 0 client.4449 ms_handle_reset on v2:172.21.15.70:6837/2915150040 2024-04-22T06:31:17.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.057+0000 7faf8be05740 1 -- 172.21.15.44:0/2690912370 >> 172.21.15.44:0/2690912370 conn(0x55e25af50ca0 msgr2=0x55e25afa7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:17.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.057+0000 7faf8be05740 1 -- 172.21.15.44:0/2690912370 shutdown_connections 2024-04-22T06:31:17.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.057+0000 7faf8be05740 1 -- 172.21.15.44:0/2690912370 wait complete. 2024-04-22T06:31:17.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.057+0000 7faf8be05740 20 client.4449 trim_cache size 0 max 16384 2024-04-22T06:31:17.062 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-22T06:31:17.062 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "9b0a34ca-403d-4513-8a66-d3749ba559aa", 2024-04-22T06:31:17.062 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-04-22T06:31:17.062 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 4308, 2024-04-22T06:31:17.062 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-04-22T06:31:17.062 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-04-22T06:31:17.062 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-04-22T06:31:17.062 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 10.049379901, 2024-04-22T06:31:17.062 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 5, 2024-04-22T06:31:17.062 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 25, 2024-04-22T06:31:17.062 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 24, 2024-04-22T06:31:17.063 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 16.949303257 2024-04-22T06:31:17.063 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-22T06:31:17.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.058+0000 7faf8913b640 1 -- 172.21.15.44:0/2646801849 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7faf7006c8f0 msgr2=0x7faf7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:17.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.058+0000 7faf8913b640 1 --2- 172.21.15.44:0/2646801849 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7faf7006c8f0 0x7faf7006edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faf74000f60 tx=0x7faf74007d50 comp rx=0 tx=0).stop 2024-04-22T06:31:17.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.058+0000 7faf8913b640 1 -- 172.21.15.44:0/2646801849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf841552e0 msgr2=0x7faf8415a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:17.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.058+0000 7faf8913b640 1 --2- 172.21.15.44:0/2646801849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf841552e0 0x7faf8415a660 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7faf7c009dd0 tx=0x7faf7c00c490 comp rx=0 tx=0).stop 2024-04-22T06:31:17.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.058+0000 7faf8913b640 1 -- 172.21.15.44:0/2646801849 shutdown_connections 2024-04-22T06:31:17.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.058+0000 7faf8913b640 1 --2- 172.21.15.44:0/2646801849 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7faf7006c8f0 0x7faf7006edb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:17.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.058+0000 7faf8913b640 1 --2- 172.21.15.44:0/2646801849 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faf84154f00 0x7faf840b5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:17.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.058+0000 7faf8913b640 1 --2- 172.21.15.44:0/2646801849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf841552e0 0x7faf8415a660 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:17.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.058+0000 7faf8913b640 1 --2- 172.21.15.44:0/2646801849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf8406d470 0x7faf841f1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:17.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.058+0000 7faf8913b640 1 -- 172.21.15.44:0/2646801849 >> 172.21.15.44:0/2646801849 conn(0x7faf840733c0 msgr2=0x7faf840d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:17.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.058+0000 7faf8913b640 1 -- 172.21.15.44:0/2646801849 shutdown_connections 2024-04-22T06:31:17.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.058+0000 7faf8913b640 1 -- 172.21.15.44:0/2646801849 wait complete. 2024-04-22T06:31:17.070 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-22T06:31:17.070 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-22T06:31:17.070 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-22T06:31:17.103 INFO:tasks.ceph.mds.d.smithi070.stderr:2024-04-22T06:31:17.103+0000 7fd9169a5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-22T06:31:17.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.179+0000 7fb45e83f640 1 Processor -- start 2024-04-22T06:31:17.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.179+0000 7fb45e83f640 1 -- start start 2024-04-22T06:31:17.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.180+0000 7fb45e83f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb458155460 0x7fb45815f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:17.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.180+0000 7fb45e83f640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb4580cc6d0 0x7fb458073230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:17.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.180+0000 7fb45e83f640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4580cd0a0 con 0x7fb4580ccb40 2024-04-22T06:31:17.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.180+0000 7fb45e83f640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fb458155840 con 0x7fb4580cc6d0 2024-04-22T06:31:17.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.180+0000 7fb45e83f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fb4580d1520 con 0x7fb458155460 2024-04-22T06:31:17.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.180+0000 7fb4577fe640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb4580cc6d0 0x7fb458073230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:17.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.180+0000 7fb45cddc640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fb4580ccb40 0x7fb4580c5c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:43112/0 (socket says 172.21.15.44:43112) 2024-04-22T06:31:17.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.180+0000 7fb45cddc640 1 -- 172.21.15.44:0/3229783387 learned_addr learned my addr 172.21.15.44:0/3229783387 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:17.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.180+0000 7fb457fff640 1 --2- 172.21.15.44:0/3229783387 >> v2:172.21.15.70:3300/0 conn(0x7fb458155460 0x7fb45815f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:17.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.180+0000 7fb4577fe640 1 -- 172.21.15.44:0/3229783387 >> v2:172.21.15.70:3300/0 conn(0x7fb458155460 msgr2=0x7fb45815f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:17.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.180+0000 7fb4577fe640 1 --2- 172.21.15.44:0/3229783387 >> v2:172.21.15.70:3300/0 conn(0x7fb458155460 0x7fb45815f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:17.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.180+0000 7fb4577fe640 1 -- 172.21.15.44:0/3229783387 >> v1:172.21.15.44:6789/0 conn(0x7fb4580ccb40 legacy=0x7fb4580c5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:17.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.180+0000 7fb456ffd640 1 -- 172.21.15.44:0/3229783387 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4207061498 0 0) 0x7fb4580cd0a0 con 0x7fb4580ccb40 2024-04-22T06:31:17.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.180+0000 7fb4577fe640 1 -- 172.21.15.44:0/3229783387 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4580d0910 con 0x7fb4580cc6d0 2024-04-22T06:31:17.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.181+0000 7fb4577fe640 1 --2- 172.21.15.44:0/3229783387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4580cc6d0 0x7fb458073230 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb4580cd380 tx=0x7fb44802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31eafd782da26330 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:17.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.181+0000 7fb456ffd640 1 -- 172.21.15.44:0/3229783387 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb448035020 con 0x7fb4580cc6d0 2024-04-22T06:31:17.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.181+0000 7fb456ffd640 1 -- 172.21.15.44:0/3229783387 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb448031830 con 0x7fb4580cc6d0 2024-04-22T06:31:17.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.181+0000 7fb456ffd640 1 -- 172.21.15.44:0/3229783387 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb448031c00 con 0x7fb4580cc6d0 2024-04-22T06:31:17.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.182+0000 7fb45e83f640 1 -- 172.21.15.44:0/3229783387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4580cc6d0 msgr2=0x7fb458073230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:17.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.182+0000 7fb45e83f640 1 --2- 172.21.15.44:0/3229783387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4580cc6d0 0x7fb458073230 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb4580cd380 tx=0x7fb44802b8d0 comp rx=0 tx=0).stop 2024-04-22T06:31:17.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.182+0000 7fb45e83f640 1 -- 172.21.15.44:0/3229783387 shutdown_connections 2024-04-22T06:31:17.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.182+0000 7fb45e83f640 1 --2- 172.21.15.44:0/3229783387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4580cc6d0 0x7fb458073230 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:17.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.182+0000 7fb45e83f640 1 --2- 172.21.15.44:0/3229783387 >> v2:172.21.15.70:3300/0 conn(0x7fb458155460 0x7fb45815f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:17.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.182+0000 7fb45e83f640 1 -- 172.21.15.44:0/3229783387 >> 172.21.15.44:0/3229783387 conn(0x7fb45806cca0 msgr2=0x7fb45806d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:17.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.182+0000 7fb45e83f640 1 -- 172.21.15.44:0/3229783387 shutdown_connections 2024-04-22T06:31:17.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.182+0000 7fb45e83f640 1 -- 172.21.15.44:0/3229783387 wait complete. 2024-04-22T06:31:17.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.183+0000 7fb45e83f640 1 Processor -- start 2024-04-22T06:31:17.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.183+0000 7fb45e83f640 1 -- start start 2024-04-22T06:31:17.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.184+0000 7fb45e83f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4580cc6d0 0x7fb4581f1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:17.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.184+0000 7fb45e83f640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb4580ccb40 0x7fb4580c8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:17.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.184+0000 7fb45e83f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb458155460 0x7fb45814ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:17.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.184+0000 7fb45e83f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb4580d11b0 con 0x7fb458155460 2024-04-22T06:31:17.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.184+0000 7fb45e83f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb4580d0ed0 con 0x7fb4580cc6d0 2024-04-22T06:31:17.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.184+0000 7fb45e83f640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb4580c6310 con 0x7fb4580ccb40 2024-04-22T06:31:17.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.184+0000 7fb45cddc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb458155460 0x7fb45814ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:17.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.184+0000 7fb45cddc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb458155460 0x7fb45814ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59652/0 (socket says 172.21.15.44:59652) 2024-04-22T06:31:17.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.184+0000 7fb45cddc640 1 -- 172.21.15.44:0/377583853 learned_addr learned my addr 172.21.15.44:0/377583853 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:17.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.184+0000 7fb457fff640 1 --2- 172.21.15.44:0/377583853 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4580cc6d0 0x7fb4581f1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:17.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.184+0000 7fb4577fe640 1 --2- 172.21.15.44:0/377583853 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb4580ccb40 0x7fb4580c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:17.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.184+0000 7fb45cddc640 1 -- 172.21.15.44:0/377583853 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb4580ccb40 msgr2=0x7fb4580c8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:17.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.184+0000 7fb45cddc640 1 --2- 172.21.15.44:0/377583853 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb4580ccb40 0x7fb4580c8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:17.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.184+0000 7fb45cddc640 1 -- 172.21.15.44:0/377583853 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4580cc6d0 msgr2=0x7fb4581f1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:17.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.184+0000 7fb45cddc640 1 --2- 172.21.15.44:0/377583853 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4580cc6d0 0x7fb4581f1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:17.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.184+0000 7fb45cddc640 1 -- 172.21.15.44:0/377583853 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4581f1d60 con 0x7fb458155460 2024-04-22T06:31:17.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.184+0000 7fb45cddc640 1 --2- 172.21.15.44:0/377583853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb458155460 0x7fb45814ed10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb45000c7a0 tx=0x7fb45000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:17.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.184+0000 7fb454ff9640 1 -- 172.21.15.44:0/377583853 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb450018070 con 0x7fb458155460 2024-04-22T06:31:17.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.185+0000 7fb454ff9640 1 -- 172.21.15.44:0/377583853 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb45000a640 con 0x7fb458155460 2024-04-22T06:31:17.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.185+0000 7fb454ff9640 1 -- 172.21.15.44:0/377583853 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45000aa70 con 0x7fb458155460 2024-04-22T06:31:17.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.185+0000 7fb45e83f640 1 -- 172.21.15.44:0/377583853 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb45814c580 con 0x7fb458155460 2024-04-22T06:31:17.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.185+0000 7fb45e83f640 1 -- 172.21.15.44:0/377583853 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb45814e230 con 0x7fb458155460 2024-04-22T06:31:17.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.185+0000 7fb45e83f640 1 -- 172.21.15.44:0/377583853 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb458075f40 con 0x7fb458155460 2024-04-22T06:31:17.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.186+0000 7fb454ff9640 1 -- 172.21.15.44:0/377583853 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb45000ac10 con 0x7fb458155460 2024-04-22T06:31:17.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.186+0000 7fb454ff9640 1 --2- 172.21.15.44:0/377583853 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb43806c8f0 0x7fb43806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:17.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.187+0000 7fb454ff9640 1 -- 172.21.15.44:0/377583853 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb45008b5b0 con 0x7fb458155460 2024-04-22T06:31:17.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.187+0000 7fb457fff640 1 --2- 172.21.15.44:0/377583853 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb43806c8f0 0x7fb43806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:17.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.187+0000 7fb457fff640 1 --2- 172.21.15.44:0/377583853 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb43806c8f0 0x7fb43806edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb440006fd0 tx=0x7fb440008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:17.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.191+0000 7fb454ff9640 1 -- 172.21.15.44:0/377583853 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb458075f40 con 0x7fb458155460 2024-04-22T06:31:17.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.413+0000 7fb45e83f640 1 -- 172.21.15.44:0/377583853 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb458068cc0 con 0x7fb458155460 2024-04-22T06:31:17.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.413+0000 7fb454ff9640 1 -- 172.21.15.44:0/377583853 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+88 (secure 0 0 0) 0x7fb4581f1d60 con 0x7fb458155460 2024-04-22T06:31:17.412 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-22T06:31:17.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.415+0000 7fb45e83f640 1 -- 172.21.15.44:0/377583853 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb43806c8f0 msgr2=0x7fb43806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:17.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.415+0000 7fb45e83f640 1 --2- 172.21.15.44:0/377583853 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb43806c8f0 0x7fb43806edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb440006fd0 tx=0x7fb440008040 comp rx=0 tx=0).stop 2024-04-22T06:31:17.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.415+0000 7fb45e83f640 1 -- 172.21.15.44:0/377583853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb458155460 msgr2=0x7fb45814ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:17.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.415+0000 7fb45e83f640 1 --2- 172.21.15.44:0/377583853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb458155460 0x7fb45814ed10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb45000c7a0 tx=0x7fb45000cc60 comp rx=0 tx=0).stop 2024-04-22T06:31:17.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.415+0000 7fb45e83f640 1 -- 172.21.15.44:0/377583853 shutdown_connections 2024-04-22T06:31:17.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.415+0000 7fb45e83f640 1 --2- 172.21.15.44:0/377583853 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb43806c8f0 0x7fb43806edb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:17.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.415+0000 7fb45e83f640 1 --2- 172.21.15.44:0/377583853 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb4580ccb40 0x7fb4580c8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:17.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.415+0000 7fb45e83f640 1 --2- 172.21.15.44:0/377583853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb458155460 0x7fb45814ed10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:17.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.415+0000 7fb45e83f640 1 --2- 172.21.15.44:0/377583853 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4580cc6d0 0x7fb4581f1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:17.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.415+0000 7fb45e83f640 1 -- 172.21.15.44:0/377583853 >> 172.21.15.44:0/377583853 conn(0x7fb45806cca0 msgr2=0x7fb4580c59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:17.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.415+0000 7fb45e83f640 1 -- 172.21.15.44:0/377583853 shutdown_connections 2024-04-22T06:31:17.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17.415+0000 7fb45e83f640 1 -- 172.21.15.44:0/377583853 wait complete. 2024-04-22T06:31:17.424 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-22T06:31:17.425 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-22T06:31:17.425 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-04-22T06:31:17.425 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-22T06:31:17.425 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-22T06:31:17.425 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-22T06:31:17.425 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-22T06:31:17.425 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-22T06:31:17.425 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-22T06:31:17.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T06:31:17.425 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-04-22T06:31:17.440 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-22T06:31:17.441 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-22T06:31:17.441 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-04-22T06:31:17.441 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-04-22T06:31:17.441 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-04-22T06:31:17.441 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-04-22T06:31:17.441 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-22T06:31:17.441 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-04-22T06:31:17.441 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-04-22T06:31:17.441 INFO:teuthology.orchestra.run.smithi044.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-22T06:31:17.441 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-04-22T06:31:17.441 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-04-22T06:31:17.441 INFO:teuthology.orchestra.run.smithi044.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-22T06:31:17.442 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-04-22T06:31:17.442 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-04-22T06:31:17.442 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-22T06:31:17.442 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 42714sec preferred_lft 42714sec 2024-04-22T06:31:17.442 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link noprefixroute 2024-04-22T06:31:17.442 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-04-22T06:31:17.442 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-22T06:31:17.442 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-04-22T06:31:17.442 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-04-22T06:31:17.443 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-22T06:31:17.443 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T06:31:17.443 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-04-22T06:31:17.443 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link add name ceph-brx type bridge 2024-04-22T06:31:17.443 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip addr flush dev ceph-brx 2024-04-22T06:31:17.443 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set ceph-brx up 2024-04-22T06:31:17.443 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-22T06:31:17.443 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-04-22T06:31:17.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T06:31:17.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T06:31:17.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T06:31:17.630 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-22T06:31:17.655 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-04-22T06:31:17.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T06:31:17.657 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-04-22T06:31:17.721 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-04-22T06:31:17.722 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-22T06:31:17.722 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-22T06:31:17.722 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-22T06:31:17.722 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-22T06:31:17.722 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T06:31:17.722 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-04-22T06:31:17.723 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-22T06:31:17.723 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-22T06:31:17.723 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-22T06:31:17.723 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-04-22T06:31:17.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T06:31:17.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T06:31:17.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T06:31:17.996 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-04-22T06:31:18.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T06:31:18.012 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list-id 2024-04-22T06:31:18.067 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T06:31:18.067 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-04-22T06:31:18.067 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-22T06:31:18.068 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-22T06:31:18.068 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-04-22T06:31:18.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T06:31:18.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T06:31:18.188 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-22T06:31:18.188 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T06:31:18.189 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-04-22T06:31:18.189 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-22T06:31:18.189 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-22T06:31:18.189 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-22T06:31:18.189 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-22T06:31:18.189 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-22T06:31:18.189 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-04-22T06:31:18.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T06:31:18.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T06:31:18.319 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T06:31:18.319 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-04-22T06:31:18.320 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-04-22T06:31:18.320 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set dev brx.0 master ceph-brx 2024-04-22T06:31:18.320 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-04-22T06:31:18.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T06:31:18.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T06:31:18.399 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-22T06:31:18.399 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-22T06:31:18.399 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-22T06:31:18.414 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-22T06:31:18.414 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-22T06:31:18.414 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-22T06:31:18.469 DEBUG:teuthology.orchestra.run.smithi044:> sudo modprobe fuse 2024-04-22T06:31:18.534 DEBUG:teuthology.orchestra.run.smithi044:> cat /proc/self/mounts | awk '{print $2}' 2024-04-22T06:31:18.590 INFO:teuthology.orchestra.run.smithi044.stdout:/proc 2024-04-22T06:31:18.591 INFO:teuthology.orchestra.run.smithi044.stdout:/sys 2024-04-22T06:31:18.591 INFO:teuthology.orchestra.run.smithi044.stdout:/dev 2024-04-22T06:31:18.591 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/security 2024-04-22T06:31:18.591 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/shm 2024-04-22T06:31:18.591 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/pts 2024-04-22T06:31:18.591 INFO:teuthology.orchestra.run.smithi044.stdout:/run 2024-04-22T06:31:18.591 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/cgroup 2024-04-22T06:31:18.591 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/pstore 2024-04-22T06:31:18.591 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/bpf 2024-04-22T06:31:18.591 INFO:teuthology.orchestra.run.smithi044.stdout:/ 2024-04-22T06:31:18.591 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/selinux 2024-04-22T06:31:18.591 INFO:teuthology.orchestra.run.smithi044.stdout:/proc/sys/fs/binfmt_misc 2024-04-22T06:31:18.591 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/mqueue 2024-04-22T06:31:18.592 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/hugepages 2024-04-22T06:31:18.592 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/debug 2024-04-22T06:31:18.592 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/tracing 2024-04-22T06:31:18.592 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-sysusers.service 2024-04-22T06:31:18.592 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/config 2024-04-22T06:31:18.592 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-22T06:31:18.592 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/fuse/connections 2024-04-22T06:31:18.592 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-sysctl.service 2024-04-22T06:31:18.592 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-22T06:31:18.592 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/nfs/rpc_pipefs 2024-04-22T06:31:18.592 INFO:teuthology.orchestra.run.smithi044.stdout:/run/user/1000 2024-04-22T06:31:18.592 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph 2024-04-22T06:31:18.592 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph/osd/ceph-0 2024-04-22T06:31:18.592 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph/osd/ceph-1 2024-04-22T06:31:18.593 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph/osd/ceph-2 2024-04-22T06:31:18.593 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph/osd/ceph-3 2024-04-22T06:31:18.593 INFO:teuthology.orchestra.run.smithi044.stdout:/run/netns 2024-04-22T06:31:18.593 INFO:teuthology.orchestra.run.smithi044.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-22T06:31:18.593 INFO:teuthology.orchestra.run.smithi044.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-22T06:31:18.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T06:31:18.593 DEBUG:teuthology.orchestra.run.smithi044:> ls /sys/fs/fuse/connections 2024-04-22T06:31:18.645 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-22T06:31:18.645 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-22T06:31:18.688 DEBUG:teuthology.orchestra.run.smithi044:> sudo modprobe fuse 2024-04-22T06:31:18.712 DEBUG:teuthology.orchestra.run.smithi044:> cat /proc/self/mounts | awk '{print $2}' 2024-04-22T06:31:18.768 INFO:teuthology.orchestra.run.smithi044.stdout:/proc 2024-04-22T06:31:18.769 INFO:teuthology.orchestra.run.smithi044.stdout:/sys 2024-04-22T06:31:18.769 INFO:teuthology.orchestra.run.smithi044.stdout:/dev 2024-04-22T06:31:18.769 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/security 2024-04-22T06:31:18.769 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/shm 2024-04-22T06:31:18.769 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/pts 2024-04-22T06:31:18.769 INFO:teuthology.orchestra.run.smithi044.stdout:/run 2024-04-22T06:31:18.769 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/cgroup 2024-04-22T06:31:18.769 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/pstore 2024-04-22T06:31:18.769 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/bpf 2024-04-22T06:31:18.769 INFO:teuthology.orchestra.run.smithi044.stdout:/ 2024-04-22T06:31:18.769 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/selinux 2024-04-22T06:31:18.769 INFO:teuthology.orchestra.run.smithi044.stdout:/proc/sys/fs/binfmt_misc 2024-04-22T06:31:18.770 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/mqueue 2024-04-22T06:31:18.770 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/hugepages 2024-04-22T06:31:18.770 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/debug 2024-04-22T06:31:18.770 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/tracing 2024-04-22T06:31:18.770 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-sysusers.service 2024-04-22T06:31:18.770 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/config 2024-04-22T06:31:18.770 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-22T06:31:18.770 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/fuse/connections 2024-04-22T06:31:18.770 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-sysctl.service 2024-04-22T06:31:18.770 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-22T06:31:18.770 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/nfs/rpc_pipefs 2024-04-22T06:31:18.770 INFO:teuthology.orchestra.run.smithi044.stdout:/run/user/1000 2024-04-22T06:31:18.770 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph 2024-04-22T06:31:18.771 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph/osd/ceph-0 2024-04-22T06:31:18.771 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph/osd/ceph-1 2024-04-22T06:31:18.771 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph/osd/ceph-2 2024-04-22T06:31:18.771 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph/osd/ceph-3 2024-04-22T06:31:18.771 INFO:teuthology.orchestra.run.smithi044.stdout:/run/netns 2024-04-22T06:31:18.771 INFO:teuthology.orchestra.run.smithi044.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-22T06:31:18.771 INFO:teuthology.orchestra.run.smithi044.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-22T06:31:18.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T06:31:18.771 DEBUG:teuthology.orchestra.run.smithi044:> ls /sys/fs/fuse/connections 2024-04-22T06:31:18.785 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi044.stderr:2024-04-22T06:31:18.786+0000 7fcfa7c0d480 -1 init, newargv = 0x562c65b16dd0 newargc=15 2024-04-22T06:31:18.785 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi044.stderr:ceph-fuse[41460]: starting ceph client 2024-04-22T06:31:18.793 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi044.stderr:ceph-fuse[41460]: starting fuse 2024-04-22T06:31:18.800 INFO:teuthology.orchestra.run.smithi044.stdout:43 2024-04-22T06:31:18.801 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-22T06:31:18.801 DEBUG:teuthology.orchestra.run.smithi044:> sudo stdin-killer -- python3 -c ' 2024-04-22T06:31:18.801 DEBUG:teuthology.orchestra.run.smithi044:> import glob 2024-04-22T06:31:18.801 DEBUG:teuthology.orchestra.run.smithi044:> import re 2024-04-22T06:31:18.801 DEBUG:teuthology.orchestra.run.smithi044:> import os 2024-04-22T06:31:18.801 DEBUG:teuthology.orchestra.run.smithi044:> import subprocess 2024-04-22T06:31:18.801 DEBUG:teuthology.orchestra.run.smithi044:> 2024-04-22T06:31:18.801 DEBUG:teuthology.orchestra.run.smithi044:> def _find_admin_socket(client_name): 2024-04-22T06:31:18.801 DEBUG:teuthology.orchestra.run.smithi044:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-22T06:31:18.801 DEBUG:teuthology.orchestra.run.smithi044:> files = glob.glob(asok_path) 2024-04-22T06:31:18.802 DEBUG:teuthology.orchestra.run.smithi044:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-22T06:31:18.802 DEBUG:teuthology.orchestra.run.smithi044:> 2024-04-22T06:31:18.802 DEBUG:teuthology.orchestra.run.smithi044:> # Given a non-glob path, it better be there 2024-04-22T06:31:18.802 DEBUG:teuthology.orchestra.run.smithi044:> if "*" not in asok_path: 2024-04-22T06:31:18.802 DEBUG:teuthology.orchestra.run.smithi044:> assert(len(files) == 1) 2024-04-22T06:31:18.802 DEBUG:teuthology.orchestra.run.smithi044:> return files[0] 2024-04-22T06:31:18.802 DEBUG:teuthology.orchestra.run.smithi044:> 2024-04-22T06:31:18.802 DEBUG:teuthology.orchestra.run.smithi044:> for f in files: 2024-04-22T06:31:18.802 DEBUG:teuthology.orchestra.run.smithi044:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-22T06:31:18.802 DEBUG:teuthology.orchestra.run.smithi044:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-22T06:31:18.802 DEBUG:teuthology.orchestra.run.smithi044:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-22T06:31:18.802 DEBUG:teuthology.orchestra.run.smithi044:> contents = proc_f.read() 2024-04-22T06:31:18.802 DEBUG:teuthology.orchestra.run.smithi044:> if mountpoint in contents: 2024-04-22T06:31:18.802 DEBUG:teuthology.orchestra.run.smithi044:> return f 2024-04-22T06:31:18.803 DEBUG:teuthology.orchestra.run.smithi044:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-22T06:31:18.803 DEBUG:teuthology.orchestra.run.smithi044:> 2024-04-22T06:31:18.803 DEBUG:teuthology.orchestra.run.smithi044:> print(_find_admin_socket("client.admin")) 2024-04-22T06:31:18.803 DEBUG:teuthology.orchestra.run.smithi044:> ' 2024-04-22T06:31:18.919 INFO:teuthology.orchestra.run.smithi044.stdout:/var/run/ceph/ceph-client.admin.41460.asok 2024-04-22T06:31:18.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T06:31:18.930 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.41460.asok 2024-04-22T06:31:18.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T06:31:18.930 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.41460.asok status 2024-04-22T06:31:19.070 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-22T06:31:19.070 INFO:teuthology.orchestra.run.smithi044.stdout: "metadata": { 2024-04-22T06:31:19.070 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_sha1": "86f7587a5a09af35f0895e1a2d08527638fae697", 2024-04-22T06:31:19.070 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_version": "ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev)", 2024-04-22T06:31:19.070 INFO:teuthology.orchestra.run.smithi044.stdout: "entity_id": "admin", 2024-04-22T06:31:19.070 INFO:teuthology.orchestra.run.smithi044.stdout: "hostname": "smithi044", 2024-04-22T06:31:19.070 INFO:teuthology.orchestra.run.smithi044.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-22T06:31:19.070 INFO:teuthology.orchestra.run.smithi044.stdout: "pid": "41460", 2024-04-22T06:31:19.071 INFO:teuthology.orchestra.run.smithi044.stdout: "root": "/" 2024-04-22T06:31:19.071 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-22T06:31:19.071 INFO:teuthology.orchestra.run.smithi044.stdout: "dentry_count": 0, 2024-04-22T06:31:19.071 INFO:teuthology.orchestra.run.smithi044.stdout: "dentry_pinned_count": 0, 2024-04-22T06:31:19.071 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 4467, 2024-04-22T06:31:19.071 INFO:teuthology.orchestra.run.smithi044.stdout: "inst": { 2024-04-22T06:31:19.071 INFO:teuthology.orchestra.run.smithi044.stdout: "name": { 2024-04-22T06:31:19.071 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "client", 2024-04-22T06:31:19.071 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4467 2024-04-22T06:31:19.071 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-22T06:31:19.071 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": { 2024-04-22T06:31:19.071 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-22T06:31:19.071 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "192.168.0.1:0", 2024-04-22T06:31:19.071 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 1498350946 2024-04-22T06:31:19.071 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-22T06:31:19.071 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-22T06:31:19.072 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": { 2024-04-22T06:31:19.072 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-22T06:31:19.072 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "192.168.0.1:0", 2024-04-22T06:31:19.072 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 1498350946 2024-04-22T06:31:19.072 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-22T06:31:19.072 INFO:teuthology.orchestra.run.smithi044.stdout: "inst_str": "client.4467 192.168.0.1:0/1498350946", 2024-04-22T06:31:19.072 INFO:teuthology.orchestra.run.smithi044.stdout: "addr_str": "192.168.0.1:0/1498350946", 2024-04-22T06:31:19.072 INFO:teuthology.orchestra.run.smithi044.stdout: "inode_count": 1, 2024-04-22T06:31:19.072 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_epoch": 5, 2024-04-22T06:31:19.072 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_epoch": 25, 2024-04-22T06:31:19.072 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_epoch_barrier": 0, 2024-04-22T06:31:19.072 INFO:teuthology.orchestra.run.smithi044.stdout: "blocklisted": false, 2024-04-22T06:31:19.072 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs" 2024-04-22T06:31:19.072 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-22T06:31:19.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T06:31:19.081 DEBUG:teuthology.orchestra.run.smithi044:> stat --file-system '--printf=%T 2024-04-22T06:31:19.081 DEBUG:teuthology.orchestra.run.smithi044:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-22T06:31:19.096 INFO:teuthology.orchestra.run.smithi044.stdout:fuseblk 2024-04-22T06:31:19.097 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-22T06:31:19.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T06:31:19.097 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-22T06:31:19.162 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-22T06:31:19.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-22T06:31:19.272 INFO:teuthology.orchestra.run.smithi044.stdout:# file: . 2024-04-22T06:31:19.273 INFO:teuthology.orchestra.run.smithi044.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-22T06:31:19.273 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:31:19.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T06:31:19.279 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T06:31:19.279 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-22T06:31:19.295 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T06:31:19.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.436+0000 7f83e8950640 1 Processor -- start 2024-04-22T06:31:19.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.436+0000 7f83e8950640 1 -- start start 2024-04-22T06:31:19.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.437+0000 7f83e8950640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f83e41552a0 0x7f83e415bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:19.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.437+0000 7f83e8950640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f83e40cc4e0 0x7f83e40ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:19.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.437+0000 7f83e8950640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83e406c6e0 con 0x7f83e4155680 2024-04-22T06:31:19.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.437+0000 7f83e8950640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f83e40cb990 con 0x7f83e41552a0 2024-04-22T06:31:19.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.437+0000 7f83e8950640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f83e406c9c0 con 0x7f83e40cc4e0 2024-04-22T06:31:19.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.437+0000 7f83e259c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f83e41552a0 0x7f83e415bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:19.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.437+0000 7f83d9d9b640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f83e4155680 0x7f83e40cbdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:43126/0 (socket says 172.21.15.44:43126) 2024-04-22T06:31:19.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.437+0000 7f83d9d9b640 1 -- 172.21.15.44:0/2455393784 learned_addr learned my addr 172.21.15.44:0/2455393784 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:19.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.437+0000 7f83e2d9d640 1 --2- 172.21.15.44:0/2455393784 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f83e40cc4e0 0x7f83e40ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:19.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.437+0000 7f83e259c640 1 -- 172.21.15.44:0/2455393784 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f83e40cc4e0 msgr2=0x7f83e40ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:19.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.437+0000 7f83e259c640 1 --2- 172.21.15.44:0/2455393784 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f83e40cc4e0 0x7f83e40ccfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:19.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.437+0000 7f83e259c640 1 -- 172.21.15.44:0/2455393784 >> v1:172.21.15.44:6789/0 conn(0x7f83e4155680 legacy=0x7f83e40cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:19.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.437+0000 7f83e259c640 1 -- 172.21.15.44:0/2455393784 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83e406cca0 con 0x7f83e41552a0 2024-04-22T06:31:19.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.437+0000 7f83e1d9b640 1 -- 172.21.15.44:0/2455393784 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1698967493 0 0) 0x7f83e406c6e0 con 0x7f83e4155680 2024-04-22T06:31:19.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.437+0000 7f83e2d9d640 1 --2- 172.21.15.44:0/2455393784 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f83e40cc4e0 0x7f83e40ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:31:19.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.438+0000 7f83e259c640 1 --2- 172.21.15.44:0/2455393784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83e41552a0 0x7f83e415bb80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f83d40097a0 tx=0x7f83d402b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bba0ac5f74a97fc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:19.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.438+0000 7f83e1d9b640 1 -- 172.21.15.44:0/2455393784 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d4034070 con 0x7f83e41552a0 2024-04-22T06:31:19.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.438+0000 7f83e1d9b640 1 -- 172.21.15.44:0/2455393784 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83d4030800 con 0x7f83e41552a0 2024-04-22T06:31:19.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.438+0000 7f83e1d9b640 1 -- 172.21.15.44:0/2455393784 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d4030bd0 con 0x7f83e41552a0 2024-04-22T06:31:19.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.438+0000 7f83e8950640 1 -- 172.21.15.44:0/2455393784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83e41552a0 msgr2=0x7f83e415bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:19.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.438+0000 7f83e8950640 1 --2- 172.21.15.44:0/2455393784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83e41552a0 0x7f83e415bb80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f83d40097a0 tx=0x7f83d402b940 comp rx=0 tx=0).stop 2024-04-22T06:31:19.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.439+0000 7f83e8950640 1 -- 172.21.15.44:0/2455393784 shutdown_connections 2024-04-22T06:31:19.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.439+0000 7f83e8950640 1 --2- 172.21.15.44:0/2455393784 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f83e40cc4e0 0x7f83e40ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:19.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.439+0000 7f83e8950640 1 --2- 172.21.15.44:0/2455393784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83e41552a0 0x7f83e415bb80 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:19.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.439+0000 7f83e8950640 1 -- 172.21.15.44:0/2455393784 >> 172.21.15.44:0/2455393784 conn(0x7f83e4075820 msgr2=0x7f83e4073000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:19.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.439+0000 7f83e8950640 1 -- 172.21.15.44:0/2455393784 shutdown_connections 2024-04-22T06:31:19.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.439+0000 7f83e8950640 1 -- 172.21.15.44:0/2455393784 wait complete. 2024-04-22T06:31:19.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.440+0000 7f83e8950640 1 Processor -- start 2024-04-22T06:31:19.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.440+0000 7f83e8950640 1 -- start start 2024-04-22T06:31:19.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.441+0000 7f83e8950640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83e40cc4e0 0x7f83e4157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:19.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.441+0000 7f83e8950640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83e41552a0 0x7f83e40c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:19.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.441+0000 7f83e8950640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f83e40c65e0 0x7f83e414eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:19.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.441+0000 7f83e8950640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f83e406d540 con 0x7f83e41552a0 2024-04-22T06:31:19.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.441+0000 7f83e8950640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f83e406d260 con 0x7f83e40cc4e0 2024-04-22T06:31:19.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.441+0000 7f83e8950640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f83e40b6690 con 0x7f83e40c65e0 2024-04-22T06:31:19.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.441+0000 7f83d9d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83e41552a0 0x7f83e40c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:19.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.441+0000 7f83d9d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83e41552a0 0x7f83e40c60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59662/0 (socket says 172.21.15.44:59662) 2024-04-22T06:31:19.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.441+0000 7f83d9d9b640 1 -- 172.21.15.44:0/3691414019 learned_addr learned my addr 172.21.15.44:0/3691414019 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:19.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.441+0000 7f83e2d9d640 1 --2- 172.21.15.44:0/3691414019 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f83e40c65e0 0x7f83e414eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:19.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.441+0000 7f83e259c640 1 --2- 172.21.15.44:0/3691414019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83e40cc4e0 0x7f83e4157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:19.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.441+0000 7f83d9d9b640 1 -- 172.21.15.44:0/3691414019 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f83e40c65e0 msgr2=0x7f83e414eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:19.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.441+0000 7f83d9d9b640 1 --2- 172.21.15.44:0/3691414019 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f83e40c65e0 0x7f83e414eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:19.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.441+0000 7f83d9d9b640 1 -- 172.21.15.44:0/3691414019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83e40cc4e0 msgr2=0x7f83e4157b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:19.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.441+0000 7f83d9d9b640 1 --2- 172.21.15.44:0/3691414019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83e40cc4e0 0x7f83e4157b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:19.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.441+0000 7f83d9d9b640 1 -- 172.21.15.44:0/3691414019 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83e4158570 con 0x7f83e41552a0 2024-04-22T06:31:19.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.441+0000 7f83d9d9b640 1 --2- 172.21.15.44:0/3691414019 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83e41552a0 0x7f83e40c60a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00c400 tx=0x7f83cc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:19.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.442+0000 7f83db7fe640 1 -- 172.21.15.44:0/3691414019 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc017070 con 0x7f83e41552a0 2024-04-22T06:31:19.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.442+0000 7f83db7fe640 1 -- 172.21.15.44:0/3691414019 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83cc005ce0 con 0x7f83e41552a0 2024-04-22T06:31:19.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.442+0000 7f83e8950640 1 -- 172.21.15.44:0/3691414019 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83e40c5300 con 0x7f83e41552a0 2024-04-22T06:31:19.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.442+0000 7f83db7fe640 1 -- 172.21.15.44:0/3691414019 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc007310 con 0x7f83e41552a0 2024-04-22T06:31:19.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.442+0000 7f83e8950640 1 -- 172.21.15.44:0/3691414019 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83e40c55d0 con 0x7f83e41552a0 2024-04-22T06:31:19.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.443+0000 7f83e8950640 1 -- 172.21.15.44:0/3691414019 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83e4068cc0 con 0x7f83e41552a0 2024-04-22T06:31:19.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.443+0000 7f83db7fe640 1 -- 172.21.15.44:0/3691414019 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f83cc01a070 con 0x7f83e41552a0 2024-04-22T06:31:19.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.444+0000 7f83db7fe640 1 --2- 172.21.15.44:0/3691414019 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f83c406c8f0 0x7f83c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:19.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.444+0000 7f83db7fe640 1 -- 172.21.15.44:0/3691414019 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f83cc08b350 con 0x7f83e41552a0 2024-04-22T06:31:19.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.444+0000 7f83e259c640 1 --2- 172.21.15.44:0/3691414019 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f83c406c8f0 0x7f83c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:19.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.444+0000 7f83e259c640 1 --2- 172.21.15.44:0/3691414019 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f83c406c8f0 0x7f83c406edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f83d4036000 tx=0x7f83d403b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:19.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.448+0000 7f83db7fe640 1 -- 172.21.15.44:0/3691414019 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f83cc0593b0 con 0x7f83e41552a0 2024-04-22T06:31:19.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.634+0000 7f83e8950640 1 -- 172.21.15.44:0/3691414019 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f83e40c9f60 con 0x7f83e41552a0 2024-04-22T06:31:19.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.635+0000 7f83db7fe640 1 -- 172.21.15.44:0/3691414019 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18466 (secure 0 0 0) 0x7f83cc058cb0 con 0x7f83e41552a0 2024-04-22T06:31:19.634 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:31:19.634 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"fsid":"9b0a34ca-403d-4513-8a66-d3749ba559aa","created":"2024-04-22T06:30:42.334581+0000","modified":"2024-04-22T06:31:11.406256+0000","last_up_change":"2024-04-22T06:30:52.942257+0000","last_in_change":"2024-04-22T06:30:44.467727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T06:30:54.560256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T06:31:04.612387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T06:31:05.312124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-22T06:31:09.025550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b4a6692-53c9-472d-b2f3-1dd29a6679ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6817","nonce":2220268493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6819","nonce":2220268493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6823","nonce":2220268493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6821","nonce":2220268493}]},"public_addr":"172.21.15.44:6817/2220268493","cluster_addr":"172.21.15.44:6819/2220268493","heartbeat_back_addr":"172.21.15.44:6823/2220268493","heartbeat_front_addr":"172.21.15.44:6821/2220268493","state":["exists","up"]},{"osd":1,"uuid":"70a6370a-7c0a-4d64-a0e4-b2e726b4f291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6809","nonce":1278678977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6811","nonce":1278678977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6815","nonce":1278678977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6813","nonce":1278678977}]},"public_addr":"172.21.15.44:6809/1278678977","cluster_addr":"172.21.15.44:6811/1278678977","heartbeat_back_addr":"172.21.15.44:6815/1278678977","heartbeat_front_addr":"172.21.15.44:6813/1278678977","state":["exists","up"]},{"osd":2,"uuid":"f2e51a62-ca91-470d-bd5a-5710c61deee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6801","nonce":2544087903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6803","nonce":2544087903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6807","nonce":2544087903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6805","nonce":2544087903}]},"public_addr":"172.21.15.44:6801/2544087903","cluster_addr":"172.21.15.44:6803/2544087903","heartbeat_back_addr":"172.21.15.44:6807/2544087903","heartbeat_front_addr":"172.21.15.44:6805/2544087903","state":["exists","up"]},{"osd":3,"uuid":"bc514203-9af9-4f1f-b18e-765bdd5de6e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6825","nonce":1692928109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6827","nonce":1692928109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6831","nonce":1692928109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6829","nonce":1692928109}]},"public_addr":"172.21.15.44:6825/1692928109","cluster_addr":"172.21.15.44:6827/1692928109","heartbeat_back_addr":"172.21.15.44:6831/1692928109","heartbeat_front_addr":"172.21.15.44:6829/1692928109","state":["exists","up"]},{"osd":4,"uuid":"357c439a-6e08-4c19-ac18-b68e6c890497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6820","nonce":3802577427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6824","nonce":3802577427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6831","nonce":3802577427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6828","nonce":3802577427}]},"public_addr":"172.21.15.70:6820/3802577427","cluster_addr":"172.21.15.70:6824/3802577427","heartbeat_back_addr":"172.21.15.70:6831/3802577427","heartbeat_front_addr":"172.21.15.70:6828/3802577427","state":["exists","up"]},{"osd":5,"uuid":"952fb88f-d56e-4305-bfa8-d32674332dfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6801","nonce":664757345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6803","nonce":664757345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6807","nonce":664757345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6805","nonce":664757345}]},"public_addr":"172.21.15.70:6801/664757345","cluster_addr":"172.21.15.70:6803/664757345","heartbeat_back_addr":"172.21.15.70:6807/664757345","heartbeat_front_addr":"172.21.15.70:6805/664757345","state":["exists","up"]},{"osd":6,"uuid":"74399b3d-fac2-45ab-9172-2cd95b02450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6817","nonce":1623253482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6821","nonce":1623253482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6827","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6829","nonce":1623253482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6825","nonce":1623253482}]},"public_addr":"172.21.15.70:6817/1623253482","cluster_addr":"172.21.15.70:6821/1623253482","heartbeat_back_addr":"172.21.15.70:6829/1623253482","heartbeat_front_addr":"172.21.15.70:6825/1623253482","state":["exists","up"]},{"osd":7,"uuid":"8f35508d-6fcc-43ff-8ee7-0498a7ab3b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6809","nonce":1551180300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6811","nonce":1551180300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6815","nonce":1551180300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6813","nonce":1551180300}]},"public_addr":"172.21.15.70:6809/1551180300","cluster_addr":"172.21.15.70:6811/1551180300","heartbeat_back_addr":"172.21.15.70:6815/1551180300","heartbeat_front_addr":"172.21.15.70:6813/1551180300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.612502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.653756+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.521169+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.649955+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.640231+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.686638+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.642199+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.700360+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T06:31:19.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.637+0000 7f83e8950640 1 -- 172.21.15.44:0/3691414019 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f83c406c8f0 msgr2=0x7f83c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:19.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.637+0000 7f83e8950640 1 --2- 172.21.15.44:0/3691414019 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f83c406c8f0 0x7f83c406edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f83d4036000 tx=0x7f83d403b040 comp rx=0 tx=0).stop 2024-04-22T06:31:19.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.637+0000 7f83e8950640 1 -- 172.21.15.44:0/3691414019 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83e41552a0 msgr2=0x7f83e40c60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:19.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.637+0000 7f83e8950640 1 --2- 172.21.15.44:0/3691414019 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83e41552a0 0x7f83e40c60a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00c400 tx=0x7f83cc00c8c0 comp rx=0 tx=0).stop 2024-04-22T06:31:19.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.637+0000 7f83e8950640 1 -- 172.21.15.44:0/3691414019 shutdown_connections 2024-04-22T06:31:19.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.637+0000 7f83e8950640 1 --2- 172.21.15.44:0/3691414019 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f83c406c8f0 0x7f83c406edb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:19.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.637+0000 7f83e8950640 1 --2- 172.21.15.44:0/3691414019 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f83e40c65e0 0x7f83e414eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:19.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.637+0000 7f83e8950640 1 --2- 172.21.15.44:0/3691414019 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83e41552a0 0x7f83e40c60a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:19.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.637+0000 7f83e8950640 1 --2- 172.21.15.44:0/3691414019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83e40cc4e0 0x7f83e4157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:19.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.637+0000 7f83e8950640 1 -- 172.21.15.44:0/3691414019 >> 172.21.15.44:0/3691414019 conn(0x7f83e4075820 msgr2=0x7f83e40c8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:19.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.637+0000 7f83e8950640 1 -- 172.21.15.44:0/3691414019 shutdown_connections 2024-04-22T06:31:19.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.637+0000 7f83e8950640 1 -- 172.21.15.44:0/3691414019 wait complete. 2024-04-22T06:31:19.647 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T06:30:54.560256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T06:31:04.612387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T06:31:05.312124+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-22T06:31:09.025550+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-22T06:31:19.647 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T06:31:19.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.751+0000 7f3c830e0640 1 Processor -- start 2024-04-22T06:31:19.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.752+0000 7f3c830e0640 1 -- start start 2024-04-22T06:31:19.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.752+0000 7f3c830e0640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3c7c0ca710 0x7f3c7c0d09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:19.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.752+0000 7f3c830e0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3c7c15be00 0x7f3c7c0d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:19.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.752+0000 7f3c830e0640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c7c0cb8c0 con 0x7f3c7c0cc860 2024-04-22T06:31:19.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.752+0000 7f3c830e0640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f3c7c15c360 con 0x7f3c7c0ca710 2024-04-22T06:31:19.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.752+0000 7f3c830e0640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f3c7c0b3b00 con 0x7f3c7c15be00 2024-04-22T06:31:19.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.752+0000 7f3c80e7c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3c7c0ca710 0x7f3c7c0d09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:19.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.752+0000 7f3c8167d640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f3c7c0cc860 0x7f3c7c0b4570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:43140/0 (socket says 172.21.15.44:43140) 2024-04-22T06:31:19.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.752+0000 7f3c8167d640 1 -- 172.21.15.44:0/2544137261 learned_addr learned my addr 172.21.15.44:0/2544137261 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:19.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.753+0000 7f3c7bfff640 1 --2- 172.21.15.44:0/2544137261 >> v2:172.21.15.70:3300/0 conn(0x7f3c7c15be00 0x7f3c7c0d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:19.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.753+0000 7f3c80e7c640 1 -- 172.21.15.44:0/2544137261 >> v2:172.21.15.70:3300/0 conn(0x7f3c7c15be00 msgr2=0x7f3c7c0d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:19.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.753+0000 7f3c80e7c640 1 --2- 172.21.15.44:0/2544137261 >> v2:172.21.15.70:3300/0 conn(0x7f3c7c15be00 0x7f3c7c0d0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:19.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.753+0000 7f3c80e7c640 1 -- 172.21.15.44:0/2544137261 >> v1:172.21.15.44:6789/0 conn(0x7f3c7c0cc860 legacy=0x7f3c7c0b4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:19.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.753+0000 7f3c80e7c640 1 -- 172.21.15.44:0/2544137261 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c7c0cbb60 con 0x7f3c7c0ca710 2024-04-22T06:31:19.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.754+0000 7f3c80e7c640 1 --2- 172.21.15.44:0/2544137261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c7c0ca710 0x7f3c7c0d09a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c0b3c80 tx=0x7f3c7002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c96397062dc711 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:19.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.754+0000 7f3c7b7fe640 1 -- 172.21.15.44:0/2544137261 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c70035020 con 0x7f3c7c0ca710 2024-04-22T06:31:19.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.754+0000 7f3c7b7fe640 1 -- 172.21.15.44:0/2544137261 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c70031830 con 0x7f3c7c0ca710 2024-04-22T06:31:19.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.754+0000 7f3c7b7fe640 1 -- 172.21.15.44:0/2544137261 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c70031c00 con 0x7f3c7c0ca710 2024-04-22T06:31:19.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.754+0000 7f3c830e0640 1 -- 172.21.15.44:0/2544137261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c7c0ca710 msgr2=0x7f3c7c0d09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:19.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.754+0000 7f3c830e0640 1 --2- 172.21.15.44:0/2544137261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c7c0ca710 0x7f3c7c0d09a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c0b3c80 tx=0x7f3c7002b8d0 comp rx=0 tx=0).stop 2024-04-22T06:31:19.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.754+0000 7f3c830e0640 1 -- 172.21.15.44:0/2544137261 shutdown_connections 2024-04-22T06:31:19.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.754+0000 7f3c830e0640 1 --2- 172.21.15.44:0/2544137261 >> v2:172.21.15.70:3300/0 conn(0x7f3c7c15be00 0x7f3c7c0d0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:19.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.754+0000 7f3c830e0640 1 --2- 172.21.15.44:0/2544137261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c7c0ca710 0x7f3c7c0d09a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:19.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.754+0000 7f3c830e0640 1 -- 172.21.15.44:0/2544137261 >> 172.21.15.44:0/2544137261 conn(0x7f3c7c06d400 msgr2=0x7f3c7c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:19.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.754+0000 7f3c830e0640 1 -- 172.21.15.44:0/2544137261 shutdown_connections 2024-04-22T06:31:19.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.754+0000 7f3c830e0640 1 -- 172.21.15.44:0/2544137261 wait complete. 2024-04-22T06:31:19.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.755+0000 7f3c830e0640 1 Processor -- start 2024-04-22T06:31:19.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.755+0000 7f3c830e0640 1 -- start start 2024-04-22T06:31:19.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.756+0000 7f3c830e0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c7c0ca710 0x7f3c7c15d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:19.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.756+0000 7f3c830e0640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3c7c0cc860 0x7f3c7c1583d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:19.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.756+0000 7f3c830e0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c7c15be00 0x7f3c7c158910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:19.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.756+0000 7f3c830e0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c0b3670 con 0x7f3c7c15be00 2024-04-22T06:31:19.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.756+0000 7f3c830e0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c0b4030 con 0x7f3c7c0ca710 2024-04-22T06:31:19.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.756+0000 7f3c830e0640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3c7c0caaf0 con 0x7f3c7c0cc860 2024-04-22T06:31:19.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.756+0000 7f3c8167d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c7c15be00 0x7f3c7c158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:19.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.756+0000 7f3c8167d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c7c15be00 0x7f3c7c158910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59678/0 (socket says 172.21.15.44:59678) 2024-04-22T06:31:19.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.756+0000 7f3c8167d640 1 -- 172.21.15.44:0/1164003175 learned_addr learned my addr 172.21.15.44:0/1164003175 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:19.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.757+0000 7f3c80e7c640 1 --2- 172.21.15.44:0/1164003175 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c7c0ca710 0x7f3c7c15d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:19.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.757+0000 7f3c7bfff640 1 --2- 172.21.15.44:0/1164003175 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3c7c0cc860 0x7f3c7c1583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:19.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.757+0000 7f3c8167d640 1 -- 172.21.15.44:0/1164003175 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3c7c0cc860 msgr2=0x7f3c7c1583d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:19.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.757+0000 7f3c8167d640 1 --2- 172.21.15.44:0/1164003175 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3c7c0cc860 0x7f3c7c1583d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:19.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.757+0000 7f3c8167d640 1 -- 172.21.15.44:0/1164003175 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c7c0ca710 msgr2=0x7f3c7c15d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:19.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.757+0000 7f3c8167d640 1 --2- 172.21.15.44:0/1164003175 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c7c0ca710 0x7f3c7c15d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:19.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.757+0000 7f3c8167d640 1 -- 172.21.15.44:0/1164003175 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c7c1f0a00 con 0x7f3c7c15be00 2024-04-22T06:31:19.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.757+0000 7f3c8167d640 1 --2- 172.21.15.44:0/1164003175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c7c15be00 0x7f3c7c158910 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3c7400cbf0 tx=0x7f3c7400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:19.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.757+0000 7f3c797fa640 1 -- 172.21.15.44:0/1164003175 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c74017070 con 0x7f3c7c15be00 2024-04-22T06:31:19.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.757+0000 7f3c797fa640 1 -- 172.21.15.44:0/1164003175 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c7400ad40 con 0x7f3c7c15be00 2024-04-22T06:31:19.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.757+0000 7f3c830e0640 1 -- 172.21.15.44:0/1164003175 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c7c15f0e0 con 0x7f3c7c15be00 2024-04-22T06:31:19.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.757+0000 7f3c797fa640 1 -- 172.21.15.44:0/1164003175 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c74012600 con 0x7f3c7c15be00 2024-04-22T06:31:19.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.757+0000 7f3c830e0640 1 -- 172.21.15.44:0/1164003175 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c7c1f0fa0 con 0x7f3c7c15be00 2024-04-22T06:31:19.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.758+0000 7f3c830e0640 1 -- 172.21.15.44:0/1164003175 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c7c068cc0 con 0x7f3c7c15be00 2024-04-22T06:31:19.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.759+0000 7f3c797fa640 1 -- 172.21.15.44:0/1164003175 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3c7401a070 con 0x7f3c7c15be00 2024-04-22T06:31:19.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.759+0000 7f3c797fa640 1 --2- 172.21.15.44:0/1164003175 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f3c6006c920 0x7f3c6006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:19.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.759+0000 7f3c797fa640 1 -- 172.21.15.44:0/1164003175 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3c7408ba60 con 0x7f3c7c15be00 2024-04-22T06:31:19.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.759+0000 7f3c80e7c640 1 --2- 172.21.15.44:0/1164003175 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f3c6006c920 0x7f3c6006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:19.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.760+0000 7f3c80e7c640 1 --2- 172.21.15.44:0/1164003175 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f3c6006c920 0x7f3c6006ede0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3c70036000 tx=0x7f3c70008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:19.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.763+0000 7f3c797fa640 1 -- 172.21.15.44:0/1164003175 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3c74059590 con 0x7f3c7c15be00 2024-04-22T06:31:19.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.947+0000 7f3c830e0640 1 -- 172.21.15.44:0/1164003175 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3c7c0c3f30 con 0x7f3c7c15be00 2024-04-22T06:31:19.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.948+0000 7f3c797fa640 1 -- 172.21.15.44:0/1164003175 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f3c7405d890 con 0x7f3c7c15be00 2024-04-22T06:31:19.947 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-04-22T06:31:19.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.950+0000 7f3c830e0640 1 -- 172.21.15.44:0/1164003175 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f3c6006c920 msgr2=0x7f3c6006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:19.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.950+0000 7f3c830e0640 1 --2- 172.21.15.44:0/1164003175 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f3c6006c920 0x7f3c6006ede0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3c70036000 tx=0x7f3c70008030 comp rx=0 tx=0).stop 2024-04-22T06:31:19.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.950+0000 7f3c830e0640 1 -- 172.21.15.44:0/1164003175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c7c15be00 msgr2=0x7f3c7c158910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:19.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.950+0000 7f3c830e0640 1 --2- 172.21.15.44:0/1164003175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c7c15be00 0x7f3c7c158910 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3c7400cbf0 tx=0x7f3c7400a430 comp rx=0 tx=0).stop 2024-04-22T06:31:19.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.950+0000 7f3c830e0640 1 -- 172.21.15.44:0/1164003175 shutdown_connections 2024-04-22T06:31:19.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.950+0000 7f3c830e0640 1 --2- 172.21.15.44:0/1164003175 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f3c6006c920 0x7f3c6006ede0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:19.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.950+0000 7f3c830e0640 1 --2- 172.21.15.44:0/1164003175 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3c7c0cc860 0x7f3c7c1583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:19.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.950+0000 7f3c830e0640 1 --2- 172.21.15.44:0/1164003175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c7c15be00 0x7f3c7c158910 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:19.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.950+0000 7f3c830e0640 1 --2- 172.21.15.44:0/1164003175 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c7c0ca710 0x7f3c7c15d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:19.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.950+0000 7f3c830e0640 1 -- 172.21.15.44:0/1164003175 >> 172.21.15.44:0/1164003175 conn(0x7f3c7c06d400 msgr2=0x7f3c7c0b4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:19.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.950+0000 7f3c830e0640 1 -- 172.21.15.44:0/1164003175 shutdown_connections 2024-04-22T06:31:19.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:19.950+0000 7f3c830e0640 1 -- 172.21.15.44:0/1164003175 wait complete. 2024-04-22T06:31:19.960 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T06:31:20.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.065+0000 7f25d565c640 1 Processor -- start 2024-04-22T06:31:20.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.065+0000 7f25d565c640 1 -- start start 2024-04-22T06:31:20.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.066+0000 7f25d565c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f25d006d470 0x7f25d00cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:20.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.066+0000 7f25d565c640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25d015ccf0 con 0x7f25d01552e0 2024-04-22T06:31:20.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.066+0000 7f25d565c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25d00c5b30 con 0x7f25d0154f00 2024-04-22T06:31:20.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.066+0000 7f25d565c640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f25d00c6740 con 0x7f25d006d470 2024-04-22T06:31:20.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.066+0000 7f25c67fc640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f25d01552e0 0x7f25d00d0f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:43142/0 (socket says 172.21.15.44:43142) 2024-04-22T06:31:20.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.066+0000 7f25cf7fe640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f25d006d470 0x7f25d00cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:20.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.066+0000 7f25c67fc640 1 -- 172.21.15.44:0/564651551 learned_addr learned my addr 172.21.15.44:0/564651551 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:20.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.066+0000 7f25cf7fe640 1 -- 172.21.15.44:0/564651551 >> v1:172.21.15.70:6789/0 conn(0x7f25d0154f00 legacy=0x7f25d015c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:20.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.066+0000 7f25cf7fe640 1 -- 172.21.15.44:0/564651551 >> v1:172.21.15.44:6789/0 conn(0x7f25d01552e0 legacy=0x7f25d00d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:20.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.066+0000 7f25ce7fc640 1 -- 172.21.15.44:0/564651551 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 878053424 0 0) 0x7f25d015ccf0 con 0x7f25d01552e0 2024-04-22T06:31:20.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.066+0000 7f25cf7fe640 1 -- 172.21.15.44:0/564651551 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25d00c5e10 con 0x7f25d006d470 2024-04-22T06:31:20.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.067+0000 7f25cf7fe640 1 --2- 172.21.15.44:0/564651551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25d006d470 0x7f25d00cb430 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f25c8009d90 tx=0x7f25c802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584ce7a460c58439 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:20.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.067+0000 7f25ce7fc640 1 -- 172.21.15.44:0/564651551 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c8039070 con 0x7f25d006d470 2024-04-22T06:31:20.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.067+0000 7f25ce7fc640 1 -- 172.21.15.44:0/564651551 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25c802cbd0 con 0x7f25d006d470 2024-04-22T06:31:20.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.067+0000 7f25ce7fc640 1 -- 172.21.15.44:0/564651551 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c8034440 con 0x7f25d006d470 2024-04-22T06:31:20.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.068+0000 7f25d565c640 1 -- 172.21.15.44:0/564651551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25d006d470 msgr2=0x7f25d00cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:20.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.068+0000 7f25d565c640 1 --2- 172.21.15.44:0/564651551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25d006d470 0x7f25d00cb430 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f25c8009d90 tx=0x7f25c802be20 comp rx=0 tx=0).stop 2024-04-22T06:31:20.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.068+0000 7f25d565c640 1 -- 172.21.15.44:0/564651551 shutdown_connections 2024-04-22T06:31:20.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.068+0000 7f25d565c640 1 --2- 172.21.15.44:0/564651551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25d006d470 0x7f25d00cb430 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:20.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.068+0000 7f25d565c640 1 -- 172.21.15.44:0/564651551 >> 172.21.15.44:0/564651551 conn(0x7f25d00733c0 msgr2=0x7f25d00737d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:20.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.068+0000 7f25d565c640 1 -- 172.21.15.44:0/564651551 shutdown_connections 2024-04-22T06:31:20.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.068+0000 7f25d565c640 1 -- 172.21.15.44:0/564651551 wait complete. 2024-04-22T06:31:20.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.069+0000 7f25d565c640 1 Processor -- start 2024-04-22T06:31:20.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.069+0000 7f25d565c640 1 -- start start 2024-04-22T06:31:20.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.070+0000 7f25d565c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25d006d470 0x7f25d01f4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:20.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.070+0000 7f25d565c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25d0154f00 0x7f25d0149e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:20.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.070+0000 7f25d565c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25d01552e0 0x7f25d014a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:20.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.070+0000 7f25d565c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f25d00c6bd0 con 0x7f25d01552e0 2024-04-22T06:31:20.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.070+0000 7f25d565c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f25d00c63d0 con 0x7f25d006d470 2024-04-22T06:31:20.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.070+0000 7f25d565c640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f25d00c6d50 con 0x7f25d0154f00 2024-04-22T06:31:20.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.070+0000 7f25cf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25d01552e0 0x7f25d014a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:20.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.070+0000 7f25cf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25d01552e0 0x7f25d014a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59688/0 (socket says 172.21.15.44:59688) 2024-04-22T06:31:20.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.070+0000 7f25cf7fe640 1 -- 172.21.15.44:0/1310114473 learned_addr learned my addr 172.21.15.44:0/1310114473 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:20.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.070+0000 7f25ceffd640 1 --2- 172.21.15.44:0/1310114473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25d006d470 0x7f25d01f4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:20.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.070+0000 7f25c67fc640 1 --2- 172.21.15.44:0/1310114473 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25d0154f00 0x7f25d0149e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:20.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.070+0000 7f25cf7fe640 1 -- 172.21.15.44:0/1310114473 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25d0154f00 msgr2=0x7f25d0149e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:20.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.070+0000 7f25cf7fe640 1 --2- 172.21.15.44:0/1310114473 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25d0154f00 0x7f25d0149e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:20.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.070+0000 7f25cf7fe640 1 -- 172.21.15.44:0/1310114473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25d006d470 msgr2=0x7f25d01f4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:20.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.070+0000 7f25cf7fe640 1 --2- 172.21.15.44:0/1310114473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25d006d470 0x7f25d01f4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:20.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.070+0000 7f25cf7fe640 1 -- 172.21.15.44:0/1310114473 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25d01f4f60 con 0x7f25d01552e0 2024-04-22T06:31:20.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.070+0000 7f25cf7fe640 1 --2- 172.21.15.44:0/1310114473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25d01552e0 0x7f25d014a3d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f25c8009eb0 tx=0x7f25c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:20.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.071+0000 7f25c7fff640 1 -- 172.21.15.44:0/1310114473 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c8044020 con 0x7f25d01552e0 2024-04-22T06:31:20.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.071+0000 7f25c7fff640 1 -- 172.21.15.44:0/1310114473 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25c803f470 con 0x7f25d01552e0 2024-04-22T06:31:20.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.071+0000 7f25d565c640 1 -- 172.21.15.44:0/1310114473 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25d00cc130 con 0x7f25d01552e0 2024-04-22T06:31:20.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.071+0000 7f25c7fff640 1 -- 172.21.15.44:0/1310114473 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c8039040 con 0x7f25d01552e0 2024-04-22T06:31:20.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.071+0000 7f25d565c640 1 -- 172.21.15.44:0/1310114473 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25d00c8180 con 0x7f25d01552e0 2024-04-22T06:31:20.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.072+0000 7f25d565c640 1 -- 172.21.15.44:0/1310114473 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25d0068cc0 con 0x7f25d01552e0 2024-04-22T06:31:20.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.072+0000 7f25c7fff640 1 -- 172.21.15.44:0/1310114473 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f25c8047020 con 0x7f25d01552e0 2024-04-22T06:31:20.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.073+0000 7f25c7fff640 1 --2- 172.21.15.44:0/1310114473 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f25b006c8f0 0x7f25b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:20.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.073+0000 7f25c7fff640 1 -- 172.21.15.44:0/1310114473 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f25c80ae9f0 con 0x7f25d01552e0 2024-04-22T06:31:20.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.073+0000 7f25ceffd640 1 --2- 172.21.15.44:0/1310114473 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f25b006c8f0 0x7f25b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:20.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.073+0000 7f25ceffd640 1 --2- 172.21.15.44:0/1310114473 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f25b006c8f0 0x7f25b006edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f25c0004f60 tx=0x7f25c0007e30 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:20.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.077+0000 7f25c7fff640 1 -- 172.21.15.44:0/1310114473 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f25c804e030 con 0x7f25d01552e0 2024-04-22T06:31:20.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.260+0000 7f25d565c640 1 -- 172.21.15.44:0/1310114473 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f25d00caee0 con 0x7f25d01552e0 2024-04-22T06:31:20.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.261+0000 7f25c7fff640 1 -- 172.21.15.44:0/1310114473 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f25c807c400 con 0x7f25d01552e0 2024-04-22T06:31:20.260 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-04-22T06:31:20.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.263+0000 7f25d565c640 1 -- 172.21.15.44:0/1310114473 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f25b006c8f0 msgr2=0x7f25b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:20.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.263+0000 7f25d565c640 1 --2- 172.21.15.44:0/1310114473 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f25b006c8f0 0x7f25b006edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f25c0004f60 tx=0x7f25c0007e30 comp rx=0 tx=0).stop 2024-04-22T06:31:20.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.263+0000 7f25d565c640 1 -- 172.21.15.44:0/1310114473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25d01552e0 msgr2=0x7f25d014a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:20.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.263+0000 7f25d565c640 1 --2- 172.21.15.44:0/1310114473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25d01552e0 0x7f25d014a3d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f25c8009eb0 tx=0x7f25c8004a40 comp rx=0 tx=0).stop 2024-04-22T06:31:20.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.263+0000 7f25d565c640 1 -- 172.21.15.44:0/1310114473 shutdown_connections 2024-04-22T06:31:20.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.263+0000 7f25d565c640 1 --2- 172.21.15.44:0/1310114473 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f25b006c8f0 0x7f25b006edb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:20.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.263+0000 7f25d565c640 1 --2- 172.21.15.44:0/1310114473 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25d0154f00 0x7f25d0149e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:20.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.263+0000 7f25d565c640 1 --2- 172.21.15.44:0/1310114473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25d01552e0 0x7f25d014a3d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:20.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.263+0000 7f25d565c640 1 --2- 172.21.15.44:0/1310114473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25d006d470 0x7f25d01f4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:20.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.263+0000 7f25d565c640 1 -- 172.21.15.44:0/1310114473 >> 172.21.15.44:0/1310114473 conn(0x7f25d00733c0 msgr2=0x7f25d014d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:20.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.263+0000 7f25d565c640 1 -- 172.21.15.44:0/1310114473 shutdown_connections 2024-04-22T06:31:20.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.263+0000 7f25d565c640 1 -- 172.21.15.44:0/1310114473 wait complete. 2024-04-22T06:31:20.272 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T06:31:20.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.377+0000 7fd776e9a640 1 Processor -- start 2024-04-22T06:31:20.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.377+0000 7fd776e9a640 1 -- start start 2024-04-22T06:31:20.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.378+0000 7fd776e9a640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7701552e0 0x7fd7700cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:20.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.378+0000 7fd776e9a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd77006d470 0x7fd7700d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:20.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.378+0000 7fd776e9a640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd77015ccf0 con 0x7fd770154f00 2024-04-22T06:31:20.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.378+0000 7fd776e9a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fd7700c6740 con 0x7fd77006d470 2024-04-22T06:31:20.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.378+0000 7fd776e9a640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd7700c5b30 con 0x7fd7701552e0 2024-04-22T06:31:20.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.378+0000 7fd774c36640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fd770154f00 0x7fd77015c5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:43158/0 (socket says 172.21.15.44:43158) 2024-04-22T06:31:20.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.378+0000 7fd774c36640 1 -- 172.21.15.44:0/3481373446 learned_addr learned my addr 172.21.15.44:0/3481373446 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:20.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.378+0000 7fd76f7fe640 1 -- 172.21.15.44:0/3481373446 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 873992960 0 0) 0x7fd77015ccf0 con 0x7fd770154f00 2024-04-22T06:31:20.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.378+0000 7fd775437640 1 --2- 172.21.15.44:0/3481373446 >> v2:172.21.15.70:3300/0 conn(0x7fd77006d470 0x7fd7700d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:20.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.378+0000 7fd76ffff640 1 --2- 172.21.15.44:0/3481373446 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7701552e0 0x7fd7700cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:20.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.378+0000 7fd76f7fe640 1 -- 172.21.15.44:0/3481373446 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd75c003520 con 0x7fd770154f00 2024-04-22T06:31:20.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.379+0000 7fd76f7fe640 1 -- 172.21.15.44:0/3481373446 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1359897973 0 0) 0x7fd75c003520 con 0x7fd770154f00 2024-04-22T06:31:20.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.379+0000 7fd76f7fe640 1 -- 172.21.15.44:0/3481373446 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7701552e0 msgr2=0x7fd7700cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:20.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.379+0000 7fd76f7fe640 1 --2- 172.21.15.44:0/3481373446 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7701552e0 0x7fd7700cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:20.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.379+0000 7fd76f7fe640 1 -- 172.21.15.44:0/3481373446 >> v2:172.21.15.70:3300/0 conn(0x7fd77006d470 msgr2=0x7fd7700d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:20.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.379+0000 7fd76f7fe640 1 --2- 172.21.15.44:0/3481373446 >> v2:172.21.15.70:3300/0 conn(0x7fd77006d470 0x7fd7700d0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:20.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.379+0000 7fd76f7fe640 1 -- 172.21.15.44:0/3481373446 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7700c5e10 con 0x7fd770154f00 2024-04-22T06:31:20.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.379+0000 7fd76f7fe640 1 -- 172.21.15.44:0/3481373446 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1237339257 0 0) 0x7fd768001670 con 0x7fd770154f00 2024-04-22T06:31:20.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.379+0000 7fd76f7fe640 1 -- 172.21.15.44:0/3481373446 >> v1:172.21.15.44:6789/0 conn(0x7fd770154f00 legacy=0x7fd77015c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:20.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.379+0000 7fd76f7fe640 1 --2- 172.21.15.44:0/3481373446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd75c0039b0 0x7fd75c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:20.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.379+0000 7fd76f7fe640 1 --2- 172.21.15.44:0/3481373446 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7701552e0 0x7fd75c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:20.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.379+0000 7fd76f7fe640 1 --2- 172.21.15.44:0/3481373446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd75c004d40 0x7fd75c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:20.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.379+0000 7fd76f7fe640 1 -- 172.21.15.44:0/3481373446 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd7700c6740 con 0x7fd75c0039b0 2024-04-22T06:31:20.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.379+0000 7fd76f7fe640 1 -- 172.21.15.44:0/3481373446 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd7700c5b30 con 0x7fd75c004d40 2024-04-22T06:31:20.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.379+0000 7fd76f7fe640 1 -- 172.21.15.44:0/3481373446 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd75c00b9c0 con 0x7fd7701552e0 2024-04-22T06:31:20.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.379+0000 7fd774c36640 1 --2- 172.21.15.44:0/3481373446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd75c0039b0 0x7fd75c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:20.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.379+0000 7fd76ffff640 1 --2- 172.21.15.44:0/3481373446 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7701552e0 0x7fd75c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:20.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.379+0000 7fd775437640 1 --2- 172.21.15.44:0/3481373446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd75c004d40 0x7fd75c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:20.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.379+0000 7fd774c36640 1 -- 172.21.15.44:0/3481373446 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7701552e0 msgr2=0x7fd75c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:20.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.379+0000 7fd774c36640 1 --2- 172.21.15.44:0/3481373446 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7701552e0 0x7fd75c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:20.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.379+0000 7fd774c36640 1 -- 172.21.15.44:0/3481373446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd75c004d40 msgr2=0x7fd75c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:20.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.379+0000 7fd774c36640 1 --2- 172.21.15.44:0/3481373446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd75c004d40 0x7fd75c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:20.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.379+0000 7fd774c36640 1 -- 172.21.15.44:0/3481373446 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd75c003520 con 0x7fd75c0039b0 2024-04-22T06:31:20.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.379+0000 7fd775437640 1 --2- 172.21.15.44:0/3481373446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd75c004d40 0x7fd75c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:31:20.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.380+0000 7fd774c36640 1 --2- 172.21.15.44:0/3481373446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd75c0039b0 0x7fd75c003dd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd76800c190 tx=0x7fd75c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a88fe160d9ee417c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:20.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.380+0000 7fd76f7fe640 1 -- 172.21.15.44:0/3481373446 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd768007c60 con 0x7fd75c0039b0 2024-04-22T06:31:20.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.380+0000 7fd76f7fe640 1 -- 172.21.15.44:0/3481373446 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd768030070 con 0x7fd75c0039b0 2024-04-22T06:31:20.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.380+0000 7fd76f7fe640 1 -- 172.21.15.44:0/3481373446 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd768005d30 con 0x7fd75c0039b0 2024-04-22T06:31:20.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.380+0000 7fd776e9a640 1 -- 172.21.15.44:0/3481373446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd75c0039b0 msgr2=0x7fd75c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:20.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.380+0000 7fd776e9a640 1 --2- 172.21.15.44:0/3481373446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd75c0039b0 0x7fd75c003dd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd76800c190 tx=0x7fd75c005830 comp rx=0 tx=0).stop 2024-04-22T06:31:20.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.381+0000 7fd775437640 1 -- 172.21.15.44:0/3481373446 reap_dead start 2024-04-22T06:31:20.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.381+0000 7fd776e9a640 1 -- 172.21.15.44:0/3481373446 shutdown_connections 2024-04-22T06:31:20.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.381+0000 7fd776e9a640 1 -- 172.21.15.44:0/3481373446 >> 172.21.15.44:0/3481373446 conn(0x7fd7700733c0 msgr2=0x7fd7700737d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:20.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.381+0000 7fd776e9a640 1 -- 172.21.15.44:0/3481373446 shutdown_connections 2024-04-22T06:31:20.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.381+0000 7fd776e9a640 1 -- 172.21.15.44:0/3481373446 wait complete. 2024-04-22T06:31:20.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.382+0000 7fd776e9a640 1 Processor -- start 2024-04-22T06:31:20.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.382+0000 7fd776e9a640 1 -- start start 2024-04-22T06:31:20.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.383+0000 7fd776e9a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd75c0039b0 0x7fd7701584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:20.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.383+0000 7fd776e9a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7701f4210 0x7fd77015ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:20.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.383+0000 7fd776e9a640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd770159560 0x7fd7701599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:20.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.383+0000 7fd776e9a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd7700c6bd0 con 0x7fd7701f4210 2024-04-22T06:31:20.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.383+0000 7fd776e9a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd7700c63d0 con 0x7fd75c0039b0 2024-04-22T06:31:20.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.383+0000 7fd776e9a640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd7700c6d50 con 0x7fd770159560 2024-04-22T06:31:20.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.383+0000 7fd76ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7701f4210 0x7fd77015ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:20.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.383+0000 7fd76ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7701f4210 0x7fd77015ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59706/0 (socket says 172.21.15.44:59706) 2024-04-22T06:31:20.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.383+0000 7fd76ffff640 1 -- 172.21.15.44:0/2746200471 learned_addr learned my addr 172.21.15.44:0/2746200471 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:20.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.383+0000 7fd775437640 1 --2- 172.21.15.44:0/2746200471 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd770159560 0x7fd7701599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:20.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.383+0000 7fd774c36640 1 --2- 172.21.15.44:0/2746200471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd75c0039b0 0x7fd7701584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:20.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.383+0000 7fd76ffff640 1 -- 172.21.15.44:0/2746200471 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd770159560 msgr2=0x7fd7701599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:20.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.383+0000 7fd76ffff640 1 --2- 172.21.15.44:0/2746200471 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd770159560 0x7fd7701599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:20.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.383+0000 7fd76ffff640 1 -- 172.21.15.44:0/2746200471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd75c0039b0 msgr2=0x7fd7701584f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:20.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.383+0000 7fd76ffff640 1 --2- 172.21.15.44:0/2746200471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd75c0039b0 0x7fd7701584f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:20.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.383+0000 7fd76ffff640 1 -- 172.21.15.44:0/2746200471 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd770158a30 con 0x7fd7701f4210 2024-04-22T06:31:20.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.383+0000 7fd775437640 1 --2- 172.21.15.44:0/2746200471 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd770159560 0x7fd7701599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:31:20.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.383+0000 7fd774c36640 1 --2- 172.21.15.44:0/2746200471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd75c0039b0 0x7fd7701584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:31:20.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.383+0000 7fd76ffff640 1 --2- 172.21.15.44:0/2746200471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7701f4210 0x7fd77015ee70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd760011e60 tx=0x7fd76000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:20.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.383+0000 7fd76d7fa640 1 -- 172.21.15.44:0/2746200471 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd760015630 con 0x7fd7701f4210 2024-04-22T06:31:20.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.383+0000 7fd76d7fa640 1 -- 172.21.15.44:0/2746200471 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7600157d0 con 0x7fd7701f4210 2024-04-22T06:31:20.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.383+0000 7fd76d7fa640 1 -- 172.21.15.44:0/2746200471 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd760015ba0 con 0x7fd7701f4210 2024-04-22T06:31:20.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.383+0000 7fd776e9a640 1 -- 172.21.15.44:0/2746200471 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7701f10c0 con 0x7fd7701f4210 2024-04-22T06:31:20.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.383+0000 7fd776e9a640 1 -- 172.21.15.44:0/2746200471 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7701f1680 con 0x7fd7701f4210 2024-04-22T06:31:20.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.384+0000 7fd776e9a640 1 -- 172.21.15.44:0/2746200471 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7700c74d0 con 0x7fd7701f4210 2024-04-22T06:31:20.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.385+0000 7fd76d7fa640 1 -- 172.21.15.44:0/2746200471 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd76001d400 con 0x7fd7701f4210 2024-04-22T06:31:20.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.385+0000 7fd76d7fa640 1 --2- 172.21.15.44:0/2746200471 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd75006c8f0 0x7fd75006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:20.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.386+0000 7fd76d7fa640 1 -- 172.21.15.44:0/2746200471 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd76008d030 con 0x7fd7701f4210 2024-04-22T06:31:20.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.386+0000 7fd774c36640 1 --2- 172.21.15.44:0/2746200471 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd75006c8f0 0x7fd75006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:20.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.386+0000 7fd774c36640 1 --2- 172.21.15.44:0/2746200471 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd75006c8f0 0x7fd75006edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd768034000 tx=0x7fd768007000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:20.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.390+0000 7fd76d7fa640 1 -- 172.21.15.44:0/2746200471 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd7700c74d0 con 0x7fd7701f4210 2024-04-22T06:31:20.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.574+0000 7fd776e9a640 1 -- 172.21.15.44:0/2746200471 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd770158d20 con 0x7fd7701f4210 2024-04-22T06:31:20.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.574+0000 7fd76d7fa640 1 -- 172.21.15.44:0/2746200471 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7fd770158d20 con 0x7fd7701f4210 2024-04-22T06:31:20.573 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-04-22T06:31:20.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.576+0000 7fd776e9a640 1 -- 172.21.15.44:0/2746200471 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd75006c8f0 msgr2=0x7fd75006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:20.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.576+0000 7fd776e9a640 1 --2- 172.21.15.44:0/2746200471 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd75006c8f0 0x7fd75006edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd768034000 tx=0x7fd768007000 comp rx=0 tx=0).stop 2024-04-22T06:31:20.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.576+0000 7fd776e9a640 1 -- 172.21.15.44:0/2746200471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7701f4210 msgr2=0x7fd77015ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:20.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.576+0000 7fd776e9a640 1 --2- 172.21.15.44:0/2746200471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7701f4210 0x7fd77015ee70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd760011e60 tx=0x7fd76000f580 comp rx=0 tx=0).stop 2024-04-22T06:31:20.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.576+0000 7fd776e9a640 1 -- 172.21.15.44:0/2746200471 shutdown_connections 2024-04-22T06:31:20.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.576+0000 7fd776e9a640 1 --2- 172.21.15.44:0/2746200471 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd75006c8f0 0x7fd75006edb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:20.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.576+0000 7fd776e9a640 1 --2- 172.21.15.44:0/2746200471 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd770159560 0x7fd7701599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:20.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.576+0000 7fd776e9a640 1 --2- 172.21.15.44:0/2746200471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7701f4210 0x7fd77015ee70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:20.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.576+0000 7fd776e9a640 1 --2- 172.21.15.44:0/2746200471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd75c0039b0 0x7fd7701584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:20.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.576+0000 7fd776e9a640 1 -- 172.21.15.44:0/2746200471 >> 172.21.15.44:0/2746200471 conn(0x7fd7700733c0 msgr2=0x7fd770153c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:20.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.576+0000 7fd776e9a640 1 -- 172.21.15.44:0/2746200471 shutdown_connections 2024-04-22T06:31:20.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.577+0000 7fd776e9a640 1 -- 172.21.15.44:0/2746200471 wait complete. 2024-04-22T06:31:20.586 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-22T06:31:20.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.690+0000 7f455ccba640 1 Processor -- start 2024-04-22T06:31:20.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.691+0000 7f455ccba640 1 -- start start 2024-04-22T06:31:20.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.691+0000 7f455ccba640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f45580c83f0 0x7f455815baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:20.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.691+0000 7f455ccba640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4558074cb0 0x7f4558075160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:20.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.691+0000 7f455ccba640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4558143f30 con 0x7f45581437f0 2024-04-22T06:31:20.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.691+0000 7f455ccba640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f4558144760 con 0x7f4558074cb0 2024-04-22T06:31:20.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.691+0000 7f455ccba640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f45580757b0 con 0x7f45580c83f0 2024-04-22T06:31:20.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.692+0000 7f455659c640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f45581437f0 0x7f455815f120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:43164/0 (socket says 172.21.15.44:43164) 2024-04-22T06:31:20.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.692+0000 7f455659c640 1 -- 172.21.15.44:0/4251016526 learned_addr learned my addr 172.21.15.44:0/4251016526 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:20.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.692+0000 7f4556d9d640 1 --2- 172.21.15.44:0/4251016526 >> v2:172.21.15.70:3300/0 conn(0x7f4558074cb0 0x7f4558075160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:20.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.692+0000 7f4555d9b640 1 --2- 172.21.15.44:0/4251016526 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f45580c83f0 0x7f455815baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:20.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.692+0000 7f455559a640 1 -- 172.21.15.44:0/4251016526 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 681003430 0 0) 0x7f4558143f30 con 0x7f45581437f0 2024-04-22T06:31:20.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.692+0000 7f455559a640 1 -- 172.21.15.44:0/4251016526 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4540003520 con 0x7f45581437f0 2024-04-22T06:31:20.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.692+0000 7f4556d9d640 1 -- 172.21.15.44:0/4251016526 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f45580c83f0 msgr2=0x7f455815baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:20.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.692+0000 7f4556d9d640 1 --2- 172.21.15.44:0/4251016526 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f45580c83f0 0x7f455815baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:20.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.692+0000 7f4556d9d640 1 -- 172.21.15.44:0/4251016526 >> v1:172.21.15.44:6789/0 conn(0x7f45581437f0 legacy=0x7f455815f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:20.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.692+0000 7f455559a640 1 -- 172.21.15.44:0/4251016526 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 778650043 0 0) 0x7f4540003520 con 0x7f45581437f0 2024-04-22T06:31:20.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.692+0000 7f4556d9d640 1 -- 172.21.15.44:0/4251016526 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45581441a0 con 0x7f4558074cb0 2024-04-22T06:31:20.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.693+0000 7f4556d9d640 1 --2- 172.21.15.44:0/4251016526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4558074cb0 0x7f4558075160 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4550009d90 tx=0x7f455002be00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=531629345b7f2280 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:20.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.693+0000 7f455559a640 1 -- 172.21.15.44:0/4251016526 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4550039070 con 0x7f4558074cb0 2024-04-22T06:31:20.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.693+0000 7f455559a640 1 -- 172.21.15.44:0/4251016526 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f455002c930 con 0x7f4558074cb0 2024-04-22T06:31:20.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.693+0000 7f455559a640 1 -- 172.21.15.44:0/4251016526 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4548001670 con 0x7f4558074cb0 2024-04-22T06:31:20.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.694+0000 7f455ccba640 1 -- 172.21.15.44:0/4251016526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4558074cb0 msgr2=0x7f4558075160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:20.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.694+0000 7f455ccba640 1 --2- 172.21.15.44:0/4251016526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4558074cb0 0x7f4558075160 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4550009d90 tx=0x7f455002be00 comp rx=0 tx=0).stop 2024-04-22T06:31:20.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.694+0000 7f455ccba640 1 -- 172.21.15.44:0/4251016526 shutdown_connections 2024-04-22T06:31:20.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.694+0000 7f455ccba640 1 --2- 172.21.15.44:0/4251016526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4558074cb0 0x7f4558075160 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:20.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.694+0000 7f455ccba640 1 --2- 172.21.15.44:0/4251016526 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f45580c83f0 0x7f455815baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:20.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.694+0000 7f455ccba640 1 -- 172.21.15.44:0/4251016526 >> 172.21.15.44:0/4251016526 conn(0x7f45580cd630 msgr2=0x7f45580cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:20.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.694+0000 7f455ccba640 1 -- 172.21.15.44:0/4251016526 shutdown_connections 2024-04-22T06:31:20.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.694+0000 7f455ccba640 1 -- 172.21.15.44:0/4251016526 wait complete. 2024-04-22T06:31:20.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.695+0000 7f455ccba640 1 Processor -- start 2024-04-22T06:31:20.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.695+0000 7f455ccba640 1 -- start start 2024-04-22T06:31:20.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.696+0000 7f455ccba640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4558074cb0 0x7f455815c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:20.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.696+0000 7f455ccba640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45580c83f0 0x7f455815e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:20.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.696+0000 7f455ccba640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f45581437f0 0x7f455815e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:20.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.696+0000 7f455ccba640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f455815b5d0 con 0x7f4558074cb0 2024-04-22T06:31:20.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.696+0000 7f455ccba640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f455815b2f0 con 0x7f45580c83f0 2024-04-22T06:31:20.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.696+0000 7f455ccba640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f455815c030 con 0x7f45581437f0 2024-04-22T06:31:20.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.696+0000 7f455659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4558074cb0 0x7f455815c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:20.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.696+0000 7f455659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4558074cb0 0x7f455815c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59708/0 (socket says 172.21.15.44:59708) 2024-04-22T06:31:20.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.696+0000 7f455659c640 1 -- 172.21.15.44:0/1588704694 learned_addr learned my addr 172.21.15.44:0/1588704694 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:20.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.696+0000 7f4556d9d640 1 --2- 172.21.15.44:0/1588704694 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f45581437f0 0x7f455815e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:20.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.696+0000 7f4555d9b640 1 --2- 172.21.15.44:0/1588704694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45580c83f0 0x7f455815e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:20.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.696+0000 7f455659c640 1 -- 172.21.15.44:0/1588704694 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f45581437f0 msgr2=0x7f455815e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:20.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.696+0000 7f455659c640 1 --2- 172.21.15.44:0/1588704694 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f45581437f0 0x7f455815e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:20.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.696+0000 7f455659c640 1 -- 172.21.15.44:0/1588704694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45580c83f0 msgr2=0x7f455815e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:20.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.696+0000 7f455659c640 1 --2- 172.21.15.44:0/1588704694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45580c83f0 0x7f455815e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:20.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.696+0000 7f455659c640 1 -- 172.21.15.44:0/1588704694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45580c6e80 con 0x7f4558074cb0 2024-04-22T06:31:20.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.696+0000 7f4556d9d640 1 --2- 172.21.15.44:0/1588704694 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f45581437f0 0x7f455815e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:31:20.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.696+0000 7f4555d9b640 1 --2- 172.21.15.44:0/1588704694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45580c83f0 0x7f455815e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:31:20.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.696+0000 7f455659c640 1 --2- 172.21.15.44:0/1588704694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4558074cb0 0x7f455815c310 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f454800c7b0 tx=0x7f454800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:20.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.697+0000 7f4546ffd640 1 -- 172.21.15.44:0/1588704694 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4548005ad0 con 0x7f4558074cb0 2024-04-22T06:31:20.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.697+0000 7f4546ffd640 1 -- 172.21.15.44:0/1588704694 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4548005c90 con 0x7f4558074cb0 2024-04-22T06:31:20.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.697+0000 7f455ccba640 1 -- 172.21.15.44:0/1588704694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f455813ef30 con 0x7f4558074cb0 2024-04-22T06:31:20.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.697+0000 7f4546ffd640 1 -- 172.21.15.44:0/1588704694 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45480072b0 con 0x7f4558074cb0 2024-04-22T06:31:20.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.697+0000 7f455ccba640 1 -- 172.21.15.44:0/1588704694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45580c75e0 con 0x7f4558074cb0 2024-04-22T06:31:20.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.698+0000 7f455ccba640 1 -- 172.21.15.44:0/1588704694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4558069310 con 0x7f4558074cb0 2024-04-22T06:31:20.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.698+0000 7f4546ffd640 1 -- 172.21.15.44:0/1588704694 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4548007450 con 0x7f4558074cb0 2024-04-22T06:31:20.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.698+0000 7f4546ffd640 1 --2- 172.21.15.44:0/1588704694 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f453806c920 0x7f453806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:20.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.699+0000 7f4546ffd640 1 -- 172.21.15.44:0/1588704694 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f454808bd50 con 0x7f4558074cb0 2024-04-22T06:31:20.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.699+0000 7f4555d9b640 1 --2- 172.21.15.44:0/1588704694 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f453806c920 0x7f453806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:20.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.699+0000 7f4555d9b640 1 --2- 172.21.15.44:0/1588704694 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f453806c920 0x7f453806ede0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f45581ee970 tx=0x7f454c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:20.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.703+0000 7f4546ffd640 1 -- 172.21.15.44:0/1588704694 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4548010300 con 0x7f4558074cb0 2024-04-22T06:31:20.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.891+0000 7f455ccba640 1 -- 172.21.15.44:0/1588704694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f455815af80 con 0x7f4558074cb0 2024-04-22T06:31:20.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.892+0000 7f4546ffd640 1 -- 172.21.15.44:0/1588704694 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f4548059e30 con 0x7f4558074cb0 2024-04-22T06:31:20.891 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-04-22T06:31:20.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.893+0000 7f455ccba640 1 -- 172.21.15.44:0/1588704694 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f453806c920 msgr2=0x7f453806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:20.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.893+0000 7f455ccba640 1 --2- 172.21.15.44:0/1588704694 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f453806c920 0x7f453806ede0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f45581ee970 tx=0x7f454c008040 comp rx=0 tx=0).stop 2024-04-22T06:31:20.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.893+0000 7f455ccba640 1 -- 172.21.15.44:0/1588704694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4558074cb0 msgr2=0x7f455815c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:20.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.893+0000 7f455ccba640 1 --2- 172.21.15.44:0/1588704694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4558074cb0 0x7f455815c310 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f454800c7b0 tx=0x7f454800cc70 comp rx=0 tx=0).stop 2024-04-22T06:31:20.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.893+0000 7f455ccba640 1 -- 172.21.15.44:0/1588704694 shutdown_connections 2024-04-22T06:31:20.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.893+0000 7f455ccba640 1 --2- 172.21.15.44:0/1588704694 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f453806c920 0x7f453806ede0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:20.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.893+0000 7f455ccba640 1 --2- 172.21.15.44:0/1588704694 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f45581437f0 0x7f455815e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:20.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.893+0000 7f455ccba640 1 --2- 172.21.15.44:0/1588704694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45580c83f0 0x7f455815e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:20.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.893+0000 7f455ccba640 1 --2- 172.21.15.44:0/1588704694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4558074cb0 0x7f455815c310 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:20.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.893+0000 7f455ccba640 1 -- 172.21.15.44:0/1588704694 >> 172.21.15.44:0/1588704694 conn(0x7f45580cd630 msgr2=0x7f4558141720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:20.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.894+0000 7f455ccba640 1 -- 172.21.15.44:0/1588704694 shutdown_connections 2024-04-22T06:31:20.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:20.894+0000 7f455ccba640 1 -- 172.21.15.44:0/1588704694 wait complete. 2024-04-22T06:31:20.904 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-22T06:31:21.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.007+0000 7f40833a4640 1 Processor -- start 2024-04-22T06:31:21.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.007+0000 7f40833a4640 1 -- start start 2024-04-22T06:31:21.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.008+0000 7f40833a4640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f407c1552e0 0x7f407c0cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:21.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.008+0000 7f40833a4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f407c06d470 0x7f407c0d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:21.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.008+0000 7f40833a4640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f407c15ccf0 con 0x7f407c154f00 2024-04-22T06:31:21.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.008+0000 7f40833a4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f407c0c6740 con 0x7f407c06d470 2024-04-22T06:31:21.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.008+0000 7f40833a4640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f407c0c5b30 con 0x7f407c1552e0 2024-04-22T06:31:21.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.008+0000 7f4081140640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f407c154f00 0x7f407c15c5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:43180/0 (socket says 172.21.15.44:43180) 2024-04-22T06:31:21.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.008+0000 7f4081140640 1 -- 172.21.15.44:0/2680803996 learned_addr learned my addr 172.21.15.44:0/2680803996 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:21.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.008+0000 7f4081941640 1 --2- 172.21.15.44:0/2680803996 >> v2:172.21.15.70:3300/0 conn(0x7f407c06d470 0x7f407c0d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:21.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.008+0000 7f408093f640 1 --2- 172.21.15.44:0/2680803996 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f407c1552e0 0x7f407c0cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:21.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.008+0000 7f4073fff640 1 -- 172.21.15.44:0/2680803996 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2162411888 0 0) 0x7f407c15ccf0 con 0x7f407c154f00 2024-04-22T06:31:21.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.008+0000 7f4073fff640 1 -- 172.21.15.44:0/2680803996 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4068003520 con 0x7f407c154f00 2024-04-22T06:31:21.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.008+0000 7f4073fff640 1 -- 172.21.15.44:0/2680803996 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 636443608 0 0) 0x7f4068003520 con 0x7f407c154f00 2024-04-22T06:31:21.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.008+0000 7f4073fff640 1 -- 172.21.15.44:0/2680803996 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f407c1552e0 msgr2=0x7f407c0cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:21.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.008+0000 7f4073fff640 1 --2- 172.21.15.44:0/2680803996 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f407c1552e0 0x7f407c0cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:21.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.008+0000 7f4073fff640 1 -- 172.21.15.44:0/2680803996 >> v2:172.21.15.70:3300/0 conn(0x7f407c06d470 msgr2=0x7f407c0d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:21.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.008+0000 7f4073fff640 1 --2- 172.21.15.44:0/2680803996 >> v2:172.21.15.70:3300/0 conn(0x7f407c06d470 0x7f407c0d0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:21.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.008+0000 7f4073fff640 1 -- 172.21.15.44:0/2680803996 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f407c0c5e10 con 0x7f407c154f00 2024-04-22T06:31:21.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.008+0000 7f4073fff640 1 -- 172.21.15.44:0/2680803996 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1237339257 0 0) 0x7f4078001670 con 0x7f407c154f00 2024-04-22T06:31:21.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.008+0000 7f4081941640 1 --2- 172.21.15.44:0/2680803996 >> v2:172.21.15.70:3300/0 conn(0x7f407c06d470 0x7f407c0d0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:31:21.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.008+0000 7f4073fff640 1 -- 172.21.15.44:0/2680803996 >> v1:172.21.15.44:6789/0 conn(0x7f407c154f00 legacy=0x7f407c15c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:21.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.008+0000 7f4073fff640 1 --2- 172.21.15.44:0/2680803996 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f407c1552e0 0x7f40680039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:21.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.008+0000 7f4073fff640 1 --2- 172.21.15.44:0/2680803996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4068003f30 0x7f4068004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:21.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.008+0000 7f4073fff640 1 --2- 172.21.15.44:0/2680803996 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4068004d40 0x7f40680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:21.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.008+0000 7f4073fff640 1 -- 172.21.15.44:0/2680803996 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f407c0c6740 con 0x7f4068003f30 2024-04-22T06:31:21.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.008+0000 7f4073fff640 1 -- 172.21.15.44:0/2680803996 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f407c0c5b30 con 0x7f4068004d40 2024-04-22T06:31:21.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.008+0000 7f4073fff640 1 -- 172.21.15.44:0/2680803996 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f406800b9c0 con 0x7f407c1552e0 2024-04-22T06:31:21.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.009+0000 7f408093f640 1 --2- 172.21.15.44:0/2680803996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4068003f30 0x7f4068004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:21.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.009+0000 7f4081140640 1 --2- 172.21.15.44:0/2680803996 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f407c1552e0 0x7f40680039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:21.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.009+0000 7f4081941640 1 --2- 172.21.15.44:0/2680803996 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4068004d40 0x7f40680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:21.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.009+0000 7f408093f640 1 -- 172.21.15.44:0/2680803996 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f407c1552e0 msgr2=0x7f40680039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:21.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.009+0000 7f408093f640 1 --2- 172.21.15.44:0/2680803996 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f407c1552e0 0x7f40680039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:21.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.009+0000 7f408093f640 1 -- 172.21.15.44:0/2680803996 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4068004d40 msgr2=0x7f40680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:21.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.009+0000 7f408093f640 1 --2- 172.21.15.44:0/2680803996 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4068004d40 0x7f40680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:21.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.009+0000 7f408093f640 1 -- 172.21.15.44:0/2680803996 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4068003520 con 0x7f4068003f30 2024-04-22T06:31:21.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.010+0000 7f408093f640 1 --2- 172.21.15.44:0/2680803996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4068003f30 0x7f4068004800 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f406c009ac0 tx=0x7f4068005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4627aea515d83941 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:21.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.010+0000 7f4073fff640 1 -- 172.21.15.44:0/2680803996 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406c039070 con 0x7f4068003f30 2024-04-22T06:31:21.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.010+0000 7f4073fff640 1 -- 172.21.15.44:0/2680803996 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f406c02c6b0 con 0x7f4068003f30 2024-04-22T06:31:21.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.010+0000 7f4073fff640 1 -- 172.21.15.44:0/2680803996 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406c009690 con 0x7f4068003f30 2024-04-22T06:31:21.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.010+0000 7f40833a4640 1 -- 172.21.15.44:0/2680803996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4068003f30 msgr2=0x7f4068004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:21.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.010+0000 7f40833a4640 1 --2- 172.21.15.44:0/2680803996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4068003f30 0x7f4068004800 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f406c009ac0 tx=0x7f4068005830 comp rx=0 tx=0).stop 2024-04-22T06:31:21.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.010+0000 7f4081941640 1 -- 172.21.15.44:0/2680803996 reap_dead start 2024-04-22T06:31:21.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.010+0000 7f40833a4640 1 -- 172.21.15.44:0/2680803996 shutdown_connections 2024-04-22T06:31:21.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.010+0000 7f40833a4640 1 -- 172.21.15.44:0/2680803996 >> 172.21.15.44:0/2680803996 conn(0x7f407c0733c0 msgr2=0x7f407c0737d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:21.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.010+0000 7f40833a4640 1 -- 172.21.15.44:0/2680803996 shutdown_connections 2024-04-22T06:31:21.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.010+0000 7f40833a4640 1 -- 172.21.15.44:0/2680803996 wait complete. 2024-04-22T06:31:21.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.012+0000 7f40833a4640 1 Processor -- start 2024-04-22T06:31:21.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.012+0000 7f40833a4640 1 -- start start 2024-04-22T06:31:21.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.012+0000 7f40833a4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4068003f30 0x7f407c1f1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:21.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.012+0000 7f40833a4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f407c15ef60 0x7f407c14e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:21.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.012+0000 7f40833a4640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f407c0cc640 0x7f407c0ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:21.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.012+0000 7f40833a4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f407c0c6bd0 con 0x7f4068003f30 2024-04-22T06:31:21.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.012+0000 7f40833a4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f407c0c63d0 con 0x7f407c15ef60 2024-04-22T06:31:21.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.012+0000 7f40833a4640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f407c0c6d50 con 0x7f407c0cc640 2024-04-22T06:31:21.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.012+0000 7f4081140640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4068003f30 0x7f407c1f1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:21.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.012+0000 7f4081140640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4068003f30 0x7f407c1f1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59734/0 (socket says 172.21.15.44:59734) 2024-04-22T06:31:21.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.012+0000 7f4081140640 1 -- 172.21.15.44:0/3973006485 learned_addr learned my addr 172.21.15.44:0/3973006485 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:21.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.012+0000 7f408093f640 1 --2- 172.21.15.44:0/3973006485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f407c15ef60 0x7f407c14e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:21.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.013+0000 7f4081941640 1 --2- 172.21.15.44:0/3973006485 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f407c0cc640 0x7f407c0ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:21.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.013+0000 7f4081140640 1 -- 172.21.15.44:0/3973006485 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f407c0cc640 msgr2=0x7f407c0ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:21.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.013+0000 7f4081140640 1 --2- 172.21.15.44:0/3973006485 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f407c0cc640 0x7f407c0ccac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:21.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.013+0000 7f4081140640 1 -- 172.21.15.44:0/3973006485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f407c15ef60 msgr2=0x7f407c14e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:21.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.013+0000 7f4081140640 1 --2- 172.21.15.44:0/3973006485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f407c15ef60 0x7f407c14e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:21.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.013+0000 7f4081140640 1 -- 172.21.15.44:0/3973006485 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f407c0cc400 con 0x7f4068003f30 2024-04-22T06:31:21.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.013+0000 7f4081140640 1 --2- 172.21.15.44:0/3973006485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4068003f30 0x7f407c1f1610 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f40780073a0 tx=0x7f4078007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:21.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.013+0000 7f4071ffb640 1 -- 172.21.15.44:0/3973006485 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f407801a070 con 0x7f4068003f30 2024-04-22T06:31:21.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.013+0000 7f4071ffb640 1 -- 172.21.15.44:0/3973006485 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f407800f9c0 con 0x7f4068003f30 2024-04-22T06:31:21.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.013+0000 7f4071ffb640 1 -- 172.21.15.44:0/3973006485 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f407800fdb0 con 0x7f4068003f30 2024-04-22T06:31:21.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.013+0000 7f40833a4640 1 -- 172.21.15.44:0/3973006485 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f407c15b1f0 con 0x7f4068003f30 2024-04-22T06:31:21.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.013+0000 7f40833a4640 1 -- 172.21.15.44:0/3973006485 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f407c0cc120 con 0x7f4068003f30 2024-04-22T06:31:21.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.014+0000 7f40833a4640 1 -- 172.21.15.44:0/3973006485 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f407c068cc0 con 0x7f4068003f30 2024-04-22T06:31:21.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.014+0000 7f4071ffb640 1 -- 172.21.15.44:0/3973006485 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f407801d070 con 0x7f4068003f30 2024-04-22T06:31:21.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.015+0000 7f4071ffb640 1 --2- 172.21.15.44:0/3973006485 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f406006c920 0x7f406006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:21.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.015+0000 7f4071ffb640 1 -- 172.21.15.44:0/3973006485 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f407808b810 con 0x7f4068003f30 2024-04-22T06:31:21.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.015+0000 7f408093f640 1 --2- 172.21.15.44:0/3973006485 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f406006c920 0x7f406006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:21.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.016+0000 7f408093f640 1 --2- 172.21.15.44:0/3973006485 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f406006c920 0x7f406006ede0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f406c036000 tx=0x7f406c030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:21.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.019+0000 7f4071ffb640 1 -- 172.21.15.44:0/3973006485 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4078059220 con 0x7f4068003f30 2024-04-22T06:31:21.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.216+0000 7f40833a4640 1 -- 172.21.15.44:0/3973006485 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f407c14ed30 con 0x7f4068003f30 2024-04-22T06:31:21.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.216+0000 7f4071ffb640 1 -- 172.21.15.44:0/3973006485 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7f4078089e20 con 0x7f4068003f30 2024-04-22T06:31:21.215 INFO:teuthology.orchestra.run.smithi044.stderr:listed 0 entries 2024-04-22T06:31:21.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.218+0000 7f40833a4640 1 -- 172.21.15.44:0/3973006485 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f406006c920 msgr2=0x7f406006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:21.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.218+0000 7f40833a4640 1 --2- 172.21.15.44:0/3973006485 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f406006c920 0x7f406006ede0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f406c036000 tx=0x7f406c030000 comp rx=0 tx=0).stop 2024-04-22T06:31:21.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.218+0000 7f40833a4640 1 -- 172.21.15.44:0/3973006485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4068003f30 msgr2=0x7f407c1f1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:21.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.218+0000 7f40833a4640 1 --2- 172.21.15.44:0/3973006485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4068003f30 0x7f407c1f1610 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f40780073a0 tx=0x7f4078007860 comp rx=0 tx=0).stop 2024-04-22T06:31:21.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.218+0000 7f40833a4640 1 -- 172.21.15.44:0/3973006485 shutdown_connections 2024-04-22T06:31:21.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.218+0000 7f40833a4640 1 --2- 172.21.15.44:0/3973006485 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f406006c920 0x7f406006ede0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:21.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.218+0000 7f40833a4640 1 --2- 172.21.15.44:0/3973006485 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f407c0cc640 0x7f407c0ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:21.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.218+0000 7f40833a4640 1 --2- 172.21.15.44:0/3973006485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f407c15ef60 0x7f407c14e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:21.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.218+0000 7f40833a4640 1 --2- 172.21.15.44:0/3973006485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4068003f30 0x7f407c1f1610 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:21.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.218+0000 7f40833a4640 1 -- 172.21.15.44:0/3973006485 >> 172.21.15.44:0/3973006485 conn(0x7f407c0733c0 msgr2=0x7f407c0737d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:21.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.218+0000 7f40833a4640 1 -- 172.21.15.44:0/3973006485 shutdown_connections 2024-04-22T06:31:21.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.218+0000 7f40833a4640 1 -- 172.21.15.44:0/3973006485 wait complete. 2024-04-22T06:31:21.227 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T06:31:21.227 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-22T06:31:21.243 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T06:31:21.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.385+0000 7f15ea74b640 1 Processor -- start 2024-04-22T06:31:21.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.385+0000 7f15ea74b640 1 -- start start 2024-04-22T06:31:21.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.386+0000 7f15ea74b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f15e41551d0 0x7f15e40ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:21.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.386+0000 7f15ea74b640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f15e41555b0 0x7f15e40c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:21.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.386+0000 7f15ea74b640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15e40ca8f0 con 0x7f15e4071430 2024-04-22T06:31:21.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.386+0000 7f15ea74b640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f15e406c300 con 0x7f15e41551d0 2024-04-22T06:31:21.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.386+0000 7f15ea74b640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f15e40c92f0 con 0x7f15e41555b0 2024-04-22T06:31:21.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.386+0000 7f15e3fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f15e41551d0 0x7f15e40ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:21.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.386+0000 7f15e8ce8640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f15e4071430 0x7f15e415e7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:43194/0 (socket says 172.21.15.44:43194) 2024-04-22T06:31:21.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.386+0000 7f15e8ce8640 1 -- 172.21.15.44:0/3527479100 learned_addr learned my addr 172.21.15.44:0/3527479100 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:21.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.386+0000 7f15e37fe640 1 --2- 172.21.15.44:0/3527479100 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f15e41555b0 0x7f15e40c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:21.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.387+0000 7f15e3fff640 1 -- 172.21.15.44:0/3527479100 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f15e41555b0 msgr2=0x7f15e40c9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:21.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.387+0000 7f15e3fff640 1 --2- 172.21.15.44:0/3527479100 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f15e41555b0 0x7f15e40c9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:21.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.387+0000 7f15e3fff640 1 -- 172.21.15.44:0/3527479100 >> v1:172.21.15.44:6789/0 conn(0x7f15e4071430 legacy=0x7f15e415e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:21.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.387+0000 7f15e2ffd640 1 -- 172.21.15.44:0/3527479100 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2652139088 0 0) 0x7f15e40ca8f0 con 0x7f15e4071430 2024-04-22T06:31:21.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.387+0000 7f15e3fff640 1 -- 172.21.15.44:0/3527479100 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e40ca150 con 0x7f15e41551d0 2024-04-22T06:31:21.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.387+0000 7f15e3fff640 1 --2- 172.21.15.44:0/3527479100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15e41551d0 0x7f15e40ca3b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f15d40097a0 tx=0x7f15d402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96269d5129df5a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:21.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.387+0000 7f15e2ffd640 1 -- 172.21.15.44:0/3527479100 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d4035020 con 0x7f15e41551d0 2024-04-22T06:31:21.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.388+0000 7f15e2ffd640 1 -- 172.21.15.44:0/3527479100 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15d4031830 con 0x7f15e41551d0 2024-04-22T06:31:21.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.388+0000 7f15e2ffd640 1 -- 172.21.15.44:0/3527479100 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d4031c00 con 0x7f15e41551d0 2024-04-22T06:31:21.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.388+0000 7f15ea74b640 1 -- 172.21.15.44:0/3527479100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15e41551d0 msgr2=0x7f15e40ca3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:21.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.388+0000 7f15ea74b640 1 --2- 172.21.15.44:0/3527479100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15e41551d0 0x7f15e40ca3b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f15d40097a0 tx=0x7f15d402b8d0 comp rx=0 tx=0).stop 2024-04-22T06:31:21.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.388+0000 7f15ea74b640 1 -- 172.21.15.44:0/3527479100 shutdown_connections 2024-04-22T06:31:21.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.388+0000 7f15ea74b640 1 --2- 172.21.15.44:0/3527479100 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f15e41555b0 0x7f15e40c9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:21.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.388+0000 7f15ea74b640 1 --2- 172.21.15.44:0/3527479100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15e41551d0 0x7f15e40ca3b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:21.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.388+0000 7f15ea74b640 1 -- 172.21.15.44:0/3527479100 >> 172.21.15.44:0/3527479100 conn(0x7f15e4074cb0 msgr2=0x7f15e4075090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:21.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.388+0000 7f15ea74b640 1 -- 172.21.15.44:0/3527479100 shutdown_connections 2024-04-22T06:31:21.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.388+0000 7f15ea74b640 1 -- 172.21.15.44:0/3527479100 wait complete. 2024-04-22T06:31:21.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.389+0000 7f15ea74b640 1 Processor -- start 2024-04-22T06:31:21.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.390+0000 7f15ea74b640 1 -- start start 2024-04-22T06:31:21.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.390+0000 7f15ea74b640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f15e4071430 0x7f15e414a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:21.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.390+0000 7f15ea74b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15e41551d0 0x7f15e41f2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:21.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.390+0000 7f15ea74b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15e41555b0 0x7f15e4144df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:21.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.390+0000 7f15ea74b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f15e406bcb0 con 0x7f15e41555b0 2024-04-22T06:31:21.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.390+0000 7f15ea74b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f15e406b9d0 con 0x7f15e41551d0 2024-04-22T06:31:21.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.390+0000 7f15ea74b640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f15e406bf90 con 0x7f15e4071430 2024-04-22T06:31:21.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.390+0000 7f15e8ce8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15e41555b0 0x7f15e4144df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:21.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.390+0000 7f15e8ce8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15e41555b0 0x7f15e4144df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59748/0 (socket says 172.21.15.44:59748) 2024-04-22T06:31:21.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.390+0000 7f15e8ce8640 1 -- 172.21.15.44:0/3393211624 learned_addr learned my addr 172.21.15.44:0/3393211624 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:21.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.391+0000 7f15e3fff640 1 --2- 172.21.15.44:0/3393211624 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f15e4071430 0x7f15e414a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:21.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.391+0000 7f15e37fe640 1 --2- 172.21.15.44:0/3393211624 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15e41551d0 0x7f15e41f2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:21.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.391+0000 7f15e8ce8640 1 -- 172.21.15.44:0/3393211624 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f15e4071430 msgr2=0x7f15e414a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:21.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.391+0000 7f15e8ce8640 1 --2- 172.21.15.44:0/3393211624 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f15e4071430 0x7f15e414a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:21.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.391+0000 7f15e8ce8640 1 -- 172.21.15.44:0/3393211624 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15e41551d0 msgr2=0x7f15e41f2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:21.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.391+0000 7f15e8ce8640 1 --2- 172.21.15.44:0/3393211624 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15e41551d0 0x7f15e41f2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:21.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.391+0000 7f15e8ce8640 1 -- 172.21.15.44:0/3393211624 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e414a5c0 con 0x7f15e41555b0 2024-04-22T06:31:21.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.391+0000 7f15e8ce8640 1 --2- 172.21.15.44:0/3393211624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15e41555b0 0x7f15e4144df0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f15dc00cb30 tx=0x7f15dc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:21.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.391+0000 7f15e0ff9640 1 -- 172.21.15.44:0/3393211624 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc017070 con 0x7f15e41555b0 2024-04-22T06:31:21.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.391+0000 7f15e0ff9640 1 -- 172.21.15.44:0/3393211624 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15dc00acc0 con 0x7f15e41555b0 2024-04-22T06:31:21.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.391+0000 7f15ea74b640 1 -- 172.21.15.44:0/3393211624 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15e414a7d0 con 0x7f15e41555b0 2024-04-22T06:31:21.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.391+0000 7f15e0ff9640 1 -- 172.21.15.44:0/3393211624 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc012640 con 0x7f15e41555b0 2024-04-22T06:31:21.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.391+0000 7f15ea74b640 1 -- 172.21.15.44:0/3393211624 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15e415d310 con 0x7f15e41555b0 2024-04-22T06:31:21.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.392+0000 7f15ea74b640 1 -- 172.21.15.44:0/3393211624 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15e414f250 con 0x7f15e41555b0 2024-04-22T06:31:21.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.392+0000 7f15e0ff9640 1 -- 172.21.15.44:0/3393211624 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f15dc01a070 con 0x7f15e41555b0 2024-04-22T06:31:21.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.393+0000 7f15e0ff9640 1 --2- 172.21.15.44:0/3393211624 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f15d006c8f0 0x7f15d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:21.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.393+0000 7f15e0ff9640 1 -- 172.21.15.44:0/3393211624 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f15dc08bca0 con 0x7f15e41555b0 2024-04-22T06:31:21.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.393+0000 7f15e3fff640 1 --2- 172.21.15.44:0/3393211624 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f15d006c8f0 0x7f15d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:21.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.394+0000 7f15e3fff640 1 --2- 172.21.15.44:0/3393211624 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f15d006c8f0 0x7f15d006edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f15d4036000 tx=0x7f15d4008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:21.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.397+0000 7f15e0ff9640 1 -- 172.21.15.44:0/3393211624 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f15e414f250 con 0x7f15e41555b0 2024-04-22T06:31:21.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.583+0000 7f15ea74b640 1 -- 172.21.15.44:0/3393211624 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f15e415dd50 con 0x7f15e41555b0 2024-04-22T06:31:21.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.585+0000 7f15e0ff9640 1 -- 172.21.15.44:0/3393211624 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18466 (secure 0 0 0) 0x7f15dc059d00 con 0x7f15e41555b0 2024-04-22T06:31:21.584 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:31:21.584 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"fsid":"9b0a34ca-403d-4513-8a66-d3749ba559aa","created":"2024-04-22T06:30:42.334581+0000","modified":"2024-04-22T06:31:11.406256+0000","last_up_change":"2024-04-22T06:30:52.942257+0000","last_in_change":"2024-04-22T06:30:44.467727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T06:30:54.560256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T06:31:04.612387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T06:31:05.312124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-22T06:31:09.025550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b4a6692-53c9-472d-b2f3-1dd29a6679ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6817","nonce":2220268493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6819","nonce":2220268493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6823","nonce":2220268493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6821","nonce":2220268493}]},"public_addr":"172.21.15.44:6817/2220268493","cluster_addr":"172.21.15.44:6819/2220268493","heartbeat_back_addr":"172.21.15.44:6823/2220268493","heartbeat_front_addr":"172.21.15.44:6821/2220268493","state":["exists","up"]},{"osd":1,"uuid":"70a6370a-7c0a-4d64-a0e4-b2e726b4f291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6809","nonce":1278678977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6811","nonce":1278678977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6815","nonce":1278678977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6813","nonce":1278678977}]},"public_addr":"172.21.15.44:6809/1278678977","cluster_addr":"172.21.15.44:6811/1278678977","heartbeat_back_addr":"172.21.15.44:6815/1278678977","heartbeat_front_addr":"172.21.15.44:6813/1278678977","state":["exists","up"]},{"osd":2,"uuid":"f2e51a62-ca91-470d-bd5a-5710c61deee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6801","nonce":2544087903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6803","nonce":2544087903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6807","nonce":2544087903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6805","nonce":2544087903}]},"public_addr":"172.21.15.44:6801/2544087903","cluster_addr":"172.21.15.44:6803/2544087903","heartbeat_back_addr":"172.21.15.44:6807/2544087903","heartbeat_front_addr":"172.21.15.44:6805/2544087903","state":["exists","up"]},{"osd":3,"uuid":"bc514203-9af9-4f1f-b18e-765bdd5de6e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6825","nonce":1692928109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6827","nonce":1692928109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6831","nonce":1692928109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6829","nonce":1692928109}]},"public_addr":"172.21.15.44:6825/1692928109","cluster_addr":"172.21.15.44:6827/1692928109","heartbeat_back_addr":"172.21.15.44:6831/1692928109","heartbeat_front_addr":"172.21.15.44:6829/1692928109","state":["exists","up"]},{"osd":4,"uuid":"357c439a-6e08-4c19-ac18-b68e6c890497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6820","nonce":3802577427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6824","nonce":3802577427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6831","nonce":3802577427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6828","nonce":3802577427}]},"public_addr":"172.21.15.70:6820/3802577427","cluster_addr":"172.21.15.70:6824/3802577427","heartbeat_back_addr":"172.21.15.70:6831/3802577427","heartbeat_front_addr":"172.21.15.70:6828/3802577427","state":["exists","up"]},{"osd":5,"uuid":"952fb88f-d56e-4305-bfa8-d32674332dfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6801","nonce":664757345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6803","nonce":664757345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6807","nonce":664757345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6805","nonce":664757345}]},"public_addr":"172.21.15.70:6801/664757345","cluster_addr":"172.21.15.70:6803/664757345","heartbeat_back_addr":"172.21.15.70:6807/664757345","heartbeat_front_addr":"172.21.15.70:6805/664757345","state":["exists","up"]},{"osd":6,"uuid":"74399b3d-fac2-45ab-9172-2cd95b02450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6817","nonce":1623253482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6821","nonce":1623253482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6827","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6829","nonce":1623253482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6825","nonce":1623253482}]},"public_addr":"172.21.15.70:6817/1623253482","cluster_addr":"172.21.15.70:6821/1623253482","heartbeat_back_addr":"172.21.15.70:6829/1623253482","heartbeat_front_addr":"172.21.15.70:6825/1623253482","state":["exists","up"]},{"osd":7,"uuid":"8f35508d-6fcc-43ff-8ee7-0498a7ab3b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6809","nonce":1551180300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6811","nonce":1551180300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6815","nonce":1551180300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6813","nonce":1551180300}]},"public_addr":"172.21.15.70:6809/1551180300","cluster_addr":"172.21.15.70:6811/1551180300","heartbeat_back_addr":"172.21.15.70:6815/1551180300","heartbeat_front_addr":"172.21.15.70:6813/1551180300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.612502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.653756+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.521169+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.649955+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.640231+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.686638+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.642199+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.700360+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T06:31:21.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.586+0000 7f15ea74b640 1 -- 172.21.15.44:0/3393211624 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f15d006c8f0 msgr2=0x7f15d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:21.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.586+0000 7f15ea74b640 1 --2- 172.21.15.44:0/3393211624 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f15d006c8f0 0x7f15d006edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f15d4036000 tx=0x7f15d4008030 comp rx=0 tx=0).stop 2024-04-22T06:31:21.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.586+0000 7f15ea74b640 1 -- 172.21.15.44:0/3393211624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15e41555b0 msgr2=0x7f15e4144df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:21.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.586+0000 7f15ea74b640 1 --2- 172.21.15.44:0/3393211624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15e41555b0 0x7f15e4144df0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f15dc00cb30 tx=0x7f15dc00a060 comp rx=0 tx=0).stop 2024-04-22T06:31:21.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.586+0000 7f15ea74b640 1 -- 172.21.15.44:0/3393211624 shutdown_connections 2024-04-22T06:31:21.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.586+0000 7f15ea74b640 1 --2- 172.21.15.44:0/3393211624 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f15d006c8f0 0x7f15d006edb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:21.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.587+0000 7f15ea74b640 1 --2- 172.21.15.44:0/3393211624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15e41555b0 0x7f15e4144df0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:21.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.587+0000 7f15ea74b640 1 --2- 172.21.15.44:0/3393211624 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15e41551d0 0x7f15e41f2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:21.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.587+0000 7f15ea74b640 1 --2- 172.21.15.44:0/3393211624 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f15e4071430 0x7f15e414a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:21.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.587+0000 7f15ea74b640 1 -- 172.21.15.44:0/3393211624 >> 172.21.15.44:0/3393211624 conn(0x7f15e4074cb0 msgr2=0x7f15e4153810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:21.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.587+0000 7f15ea74b640 1 -- 172.21.15.44:0/3393211624 shutdown_connections 2024-04-22T06:31:21.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.587+0000 7f15ea74b640 1 -- 172.21.15.44:0/3393211624 wait complete. 2024-04-22T06:31:21.596 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T06:30:54.560256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T06:31:04.612387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T06:31:05.312124+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-22T06:31:09.025550+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-22T06:31:21.597 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T06:31:21.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.702+0000 7fd0d467a640 1 Processor -- start 2024-04-22T06:31:21.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.702+0000 7fd0d467a640 1 -- start start 2024-04-22T06:31:21.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.703+0000 7fd0d467a640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd0cc06d470 0x7fd0cc0cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:21.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.703+0000 7fd0d467a640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0cc15ccf0 con 0x7fd0cc1552e0 2024-04-22T06:31:21.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.703+0000 7fd0d467a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0cc0c5b30 con 0x7fd0cc154f00 2024-04-22T06:31:21.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.703+0000 7fd0d467a640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fd0cc0c6740 con 0x7fd0cc06d470 2024-04-22T06:31:21.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.703+0000 7fd0d2c17640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd0cc06d470 0x7fd0cc0cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:21.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.703+0000 7fd0d1c15640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fd0cc1552e0 0x7fd0cc0d0f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:43198/0 (socket says 172.21.15.44:43198) 2024-04-22T06:31:21.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.703+0000 7fd0d1c15640 1 -- 172.21.15.44:0/134020184 learned_addr learned my addr 172.21.15.44:0/134020184 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:21.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.703+0000 7fd0d2c17640 1 -- 172.21.15.44:0/134020184 >> v1:172.21.15.70:6789/0 conn(0x7fd0cc154f00 legacy=0x7fd0cc15c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:21.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.703+0000 7fd0d2c17640 1 -- 172.21.15.44:0/134020184 >> v1:172.21.15.44:6789/0 conn(0x7fd0cc1552e0 legacy=0x7fd0cc0d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:21.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.703+0000 7fd0d1414640 1 -- 172.21.15.44:0/134020184 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 840466946 0 0) 0x7fd0cc15ccf0 con 0x7fd0cc1552e0 2024-04-22T06:31:21.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.703+0000 7fd0d2c17640 1 -- 172.21.15.44:0/134020184 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0cc0c5e10 con 0x7fd0cc06d470 2024-04-22T06:31:21.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.704+0000 7fd0d2c17640 1 --2- 172.21.15.44:0/134020184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0cc06d470 0x7fd0cc0cb430 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009d90 tx=0x7fd0bc02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2627f7db7b63a263 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:21.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.704+0000 7fd0d1414640 1 -- 172.21.15.44:0/134020184 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc039070 con 0x7fd0cc06d470 2024-04-22T06:31:21.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.704+0000 7fd0d1414640 1 -- 172.21.15.44:0/134020184 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0bc02cc60 con 0x7fd0cc06d470 2024-04-22T06:31:21.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.704+0000 7fd0d1414640 1 -- 172.21.15.44:0/134020184 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc034660 con 0x7fd0cc06d470 2024-04-22T06:31:21.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.705+0000 7fd0d467a640 1 -- 172.21.15.44:0/134020184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0cc06d470 msgr2=0x7fd0cc0cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:21.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.705+0000 7fd0d467a640 1 --2- 172.21.15.44:0/134020184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0cc06d470 0x7fd0cc0cb430 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009d90 tx=0x7fd0bc02beb0 comp rx=0 tx=0).stop 2024-04-22T06:31:21.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.705+0000 7fd0d467a640 1 -- 172.21.15.44:0/134020184 shutdown_connections 2024-04-22T06:31:21.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.705+0000 7fd0d467a640 1 --2- 172.21.15.44:0/134020184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0cc06d470 0x7fd0cc0cb430 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:21.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.705+0000 7fd0d467a640 1 -- 172.21.15.44:0/134020184 >> 172.21.15.44:0/134020184 conn(0x7fd0cc0733c0 msgr2=0x7fd0cc0737d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:21.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.705+0000 7fd0d467a640 1 -- 172.21.15.44:0/134020184 shutdown_connections 2024-04-22T06:31:21.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.705+0000 7fd0d467a640 1 -- 172.21.15.44:0/134020184 wait complete. 2024-04-22T06:31:21.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.706+0000 7fd0d467a640 1 Processor -- start 2024-04-22T06:31:21.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.706+0000 7fd0d467a640 1 -- start start 2024-04-22T06:31:21.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.707+0000 7fd0d467a640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0cc06d470 0x7fd0cc1f4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:21.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.707+0000 7fd0d467a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0cc154f00 0x7fd0cc1f0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:21.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.707+0000 7fd0d467a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0cc1552e0 0x7fd0cc0b5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:21.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.707+0000 7fd0d467a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc0c6bd0 con 0x7fd0cc1552e0 2024-04-22T06:31:21.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.707+0000 7fd0d467a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc0c63d0 con 0x7fd0cc154f00 2024-04-22T06:31:21.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.707+0000 7fd0d467a640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd0cc0c6d50 con 0x7fd0cc06d470 2024-04-22T06:31:21.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.707+0000 7fd0d2c17640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0cc1552e0 0x7fd0cc0b5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:21.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.707+0000 7fd0d2c17640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0cc1552e0 0x7fd0cc0b5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59762/0 (socket says 172.21.15.44:59762) 2024-04-22T06:31:21.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.707+0000 7fd0d2c17640 1 -- 172.21.15.44:0/3197793094 learned_addr learned my addr 172.21.15.44:0/3197793094 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:21.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.707+0000 7fd0d1c15640 1 --2- 172.21.15.44:0/3197793094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0cc154f00 0x7fd0cc1f0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:21.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.707+0000 7fd0d2416640 1 --2- 172.21.15.44:0/3197793094 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0cc06d470 0x7fd0cc1f4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:21.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.707+0000 7fd0d2c17640 1 -- 172.21.15.44:0/3197793094 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0cc06d470 msgr2=0x7fd0cc1f4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:21.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.707+0000 7fd0d2c17640 1 --2- 172.21.15.44:0/3197793094 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0cc06d470 0x7fd0cc1f4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:21.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.707+0000 7fd0d2c17640 1 -- 172.21.15.44:0/3197793094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0cc154f00 msgr2=0x7fd0cc1f0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:21.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.707+0000 7fd0d2c17640 1 --2- 172.21.15.44:0/3197793094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0cc154f00 0x7fd0cc1f0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:21.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.707+0000 7fd0d2c17640 1 -- 172.21.15.44:0/3197793094 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0cc14c760 con 0x7fd0cc1552e0 2024-04-22T06:31:21.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.707+0000 7fd0d1c15640 1 --2- 172.21.15.44:0/3197793094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0cc154f00 0x7fd0cc1f0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:31:21.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.707+0000 7fd0d2416640 1 --2- 172.21.15.44:0/3197793094 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0cc06d470 0x7fd0cc1f4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:31:21.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.708+0000 7fd0d2c17640 1 --2- 172.21.15.44:0/3197793094 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0cc1552e0 0x7fd0cc0b5750 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc037040 tx=0x7fd0bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:21.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.708+0000 7fd0caffd640 1 -- 172.21.15.44:0/3197793094 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc039050 con 0x7fd0cc1552e0 2024-04-22T06:31:21.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.708+0000 7fd0caffd640 1 -- 172.21.15.44:0/3197793094 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0bc03a040 con 0x7fd0cc1552e0 2024-04-22T06:31:21.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.708+0000 7fd0d467a640 1 -- 172.21.15.44:0/3197793094 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0cc14ca20 con 0x7fd0cc1552e0 2024-04-22T06:31:21.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.708+0000 7fd0caffd640 1 -- 172.21.15.44:0/3197793094 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc03e610 con 0x7fd0cc1552e0 2024-04-22T06:31:21.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.708+0000 7fd0d467a640 1 -- 172.21.15.44:0/3197793094 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0cc1f4470 con 0x7fd0cc1552e0 2024-04-22T06:31:21.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.709+0000 7fd0d467a640 1 -- 172.21.15.44:0/3197793094 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0cc068cc0 con 0x7fd0cc1552e0 2024-04-22T06:31:21.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.709+0000 7fd0caffd640 1 -- 172.21.15.44:0/3197793094 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd0bc047030 con 0x7fd0cc1552e0 2024-04-22T06:31:21.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.710+0000 7fd0caffd640 1 --2- 172.21.15.44:0/3197793094 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd0b806c920 0x7fd0b806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:21.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.710+0000 7fd0caffd640 1 -- 172.21.15.44:0/3197793094 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd0bc0adf20 con 0x7fd0cc1552e0 2024-04-22T06:31:21.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.710+0000 7fd0d2416640 1 --2- 172.21.15.44:0/3197793094 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd0b806c920 0x7fd0b806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:21.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.710+0000 7fd0d2416640 1 --2- 172.21.15.44:0/3197793094 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd0b806c920 0x7fd0b806ede0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4007fd0 tx=0x7fd0c4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:21.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.714+0000 7fd0caffd640 1 -- 172.21.15.44:0/3197793094 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd0bc07c0d0 con 0x7fd0cc1552e0 2024-04-22T06:31:21.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.897+0000 7fd0d467a640 1 -- 172.21.15.44:0/3197793094 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd0cc15bc00 con 0x7fd0cc1552e0 2024-04-22T06:31:21.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.897+0000 7fd0caffd640 1 -- 172.21.15.44:0/3197793094 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7fd0bc07b9d0 con 0x7fd0cc1552e0 2024-04-22T06:31:21.896 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-04-22T06:31:21.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.899+0000 7fd0d467a640 1 -- 172.21.15.44:0/3197793094 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd0b806c920 msgr2=0x7fd0b806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:21.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.899+0000 7fd0d467a640 1 --2- 172.21.15.44:0/3197793094 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd0b806c920 0x7fd0b806ede0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4007fd0 tx=0x7fd0c4009040 comp rx=0 tx=0).stop 2024-04-22T06:31:21.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.899+0000 7fd0d467a640 1 -- 172.21.15.44:0/3197793094 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0cc1552e0 msgr2=0x7fd0cc0b5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:21.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.899+0000 7fd0d467a640 1 --2- 172.21.15.44:0/3197793094 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0cc1552e0 0x7fd0cc0b5750 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc037040 tx=0x7fd0bc004a40 comp rx=0 tx=0).stop 2024-04-22T06:31:21.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.899+0000 7fd0d467a640 1 -- 172.21.15.44:0/3197793094 shutdown_connections 2024-04-22T06:31:21.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.899+0000 7fd0d467a640 1 --2- 172.21.15.44:0/3197793094 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd0b806c920 0x7fd0b806ede0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:21.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.899+0000 7fd0d467a640 1 --2- 172.21.15.44:0/3197793094 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0cc1552e0 0x7fd0cc0b5750 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:21.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.899+0000 7fd0d467a640 1 --2- 172.21.15.44:0/3197793094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0cc154f00 0x7fd0cc1f0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:21.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.899+0000 7fd0d467a640 1 --2- 172.21.15.44:0/3197793094 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0cc06d470 0x7fd0cc1f4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:21.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.899+0000 7fd0d467a640 1 -- 172.21.15.44:0/3197793094 >> 172.21.15.44:0/3197793094 conn(0x7fd0cc0733c0 msgr2=0x7fd0cc15e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:21.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.899+0000 7fd0d467a640 1 -- 172.21.15.44:0/3197793094 shutdown_connections 2024-04-22T06:31:21.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:21.899+0000 7fd0d467a640 1 -- 172.21.15.44:0/3197793094 wait complete. 2024-04-22T06:31:21.909 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T06:31:22.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.015+0000 7fa91f952640 1 Processor -- start 2024-04-22T06:31:22.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.015+0000 7fa91f952640 1 -- start start 2024-04-22T06:31:22.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.016+0000 7fa91f952640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa918154f00 0x7fa9180cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:22.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.016+0000 7fa91f952640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fa9181552e0 0x7fa91815c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:22.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.016+0000 7fa91f952640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9180c5b30 con 0x7fa91806d470 2024-04-22T06:31:22.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.016+0000 7fa91f952640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fa9180c6740 con 0x7fa9181552e0 2024-04-22T06:31:22.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.016+0000 7fa91f952640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fa9180c5e10 con 0x7fa918154f00 2024-04-22T06:31:22.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.016+0000 7fa91deef640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fa91806d470 0x7fa9180d0f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:36610/0 (socket says 172.21.15.44:36610) 2024-04-22T06:31:22.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.016+0000 7fa91ceed640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fa9181552e0 0x7fa91815c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:22.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.016+0000 7fa91deef640 1 -- 172.21.15.44:0/2380778878 learned_addr learned my addr 172.21.15.44:0/2380778878 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:22.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.017+0000 7fa91d6ee640 1 --2- 172.21.15.44:0/2380778878 >> v2:172.21.15.70:3300/0 conn(0x7fa918154f00 0x7fa9180cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:22.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.017+0000 7fa917fff640 1 -- 172.21.15.44:0/2380778878 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 345694574 0 0) 0x7fa9180c5b30 con 0x7fa91806d470 2024-04-22T06:31:22.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.017+0000 7fa917fff640 1 -- 172.21.15.44:0/2380778878 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa900003520 con 0x7fa91806d470 2024-04-22T06:31:22.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.017+0000 7fa91ceed640 1 -- 172.21.15.44:0/2380778878 >> v2:172.21.15.70:3300/0 conn(0x7fa918154f00 msgr2=0x7fa9180cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:22.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.017+0000 7fa91ceed640 1 --2- 172.21.15.44:0/2380778878 >> v2:172.21.15.70:3300/0 conn(0x7fa918154f00 0x7fa9180cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:22.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.017+0000 7fa91ceed640 1 -- 172.21.15.44:0/2380778878 >> v1:172.21.15.44:6789/0 conn(0x7fa91806d470 legacy=0x7fa9180d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:22.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.017+0000 7fa91ceed640 1 -- 172.21.15.44:0/2380778878 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9180c60f0 con 0x7fa9181552e0 2024-04-22T06:31:22.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.017+0000 7fa91ceed640 1 --2- 172.21.15.44:0/2380778878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9181552e0 0x7fa91815c5e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa91806cb60 tx=0x7fa90c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ad50697e420391 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:22.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.017+0000 7fa917fff640 1 -- 172.21.15.44:0/2380778878 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c0315a0 con 0x7fa9181552e0 2024-04-22T06:31:22.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.018+0000 7fa917fff640 1 -- 172.21.15.44:0/2380778878 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa90c037070 con 0x7fa9181552e0 2024-04-22T06:31:22.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.018+0000 7fa917fff640 1 -- 172.21.15.44:0/2380778878 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c031940 con 0x7fa9181552e0 2024-04-22T06:31:22.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.018+0000 7fa91f952640 1 -- 172.21.15.44:0/2380778878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9181552e0 msgr2=0x7fa91815c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:22.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.018+0000 7fa91f952640 1 --2- 172.21.15.44:0/2380778878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9181552e0 0x7fa91815c5e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa91806cb60 tx=0x7fa90c02b840 comp rx=0 tx=0).stop 2024-04-22T06:31:22.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.018+0000 7fa91f952640 1 -- 172.21.15.44:0/2380778878 shutdown_connections 2024-04-22T06:31:22.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.018+0000 7fa91f952640 1 --2- 172.21.15.44:0/2380778878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9181552e0 0x7fa91815c5e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:22.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.018+0000 7fa91f952640 1 --2- 172.21.15.44:0/2380778878 >> v2:172.21.15.70:3300/0 conn(0x7fa918154f00 0x7fa9180cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:22.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.018+0000 7fa91f952640 1 -- 172.21.15.44:0/2380778878 >> 172.21.15.44:0/2380778878 conn(0x7fa9180733c0 msgr2=0x7fa9180737d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:22.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.018+0000 7fa91f952640 1 -- 172.21.15.44:0/2380778878 shutdown_connections 2024-04-22T06:31:22.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.018+0000 7fa91f952640 1 -- 172.21.15.44:0/2380778878 wait complete. 2024-04-22T06:31:22.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.020+0000 7fa91f952640 1 Processor -- start 2024-04-22T06:31:22.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.020+0000 7fa91f952640 1 -- start start 2024-04-22T06:31:22.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.020+0000 7fa91f952640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa91806d470 0x7fa91814e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:22.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.020+0000 7fa91f952640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa918154f00 0x7fa918157850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:22.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.020+0000 7fa91f952640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa9181552e0 0x7fa918157d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:22.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.020+0000 7fa91f952640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa9180c6d50 con 0x7fa91806d470 2024-04-22T06:31:22.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.020+0000 7fa91f952640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa9180c6bd0 con 0x7fa918154f00 2024-04-22T06:31:22.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.020+0000 7fa91f952640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa9180d1690 con 0x7fa9181552e0 2024-04-22T06:31:22.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.020+0000 7fa91d6ee640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa91806d470 0x7fa91814e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:22.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.020+0000 7fa91d6ee640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa91806d470 0x7fa91814e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34158/0 (socket says 172.21.15.44:34158) 2024-04-22T06:31:22.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.020+0000 7fa91d6ee640 1 -- 172.21.15.44:0/1353486968 learned_addr learned my addr 172.21.15.44:0/1353486968 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:22.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.021+0000 7fa91deef640 1 --2- 172.21.15.44:0/1353486968 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa9181552e0 0x7fa918157d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:22.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.021+0000 7fa91ceed640 1 --2- 172.21.15.44:0/1353486968 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa918154f00 0x7fa918157850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:22.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.021+0000 7fa91d6ee640 1 -- 172.21.15.44:0/1353486968 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa9181552e0 msgr2=0x7fa918157d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:22.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.021+0000 7fa91d6ee640 1 --2- 172.21.15.44:0/1353486968 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa9181552e0 0x7fa918157d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:22.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.021+0000 7fa91d6ee640 1 -- 172.21.15.44:0/1353486968 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa918154f00 msgr2=0x7fa918157850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:22.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.021+0000 7fa91d6ee640 1 --2- 172.21.15.44:0/1353486968 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa918154f00 0x7fa918157850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:22.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.021+0000 7fa91d6ee640 1 -- 172.21.15.44:0/1353486968 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa918156b00 con 0x7fa91806d470 2024-04-22T06:31:22.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.021+0000 7fa91ceed640 1 --2- 172.21.15.44:0/1353486968 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa918154f00 0x7fa918157850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:31:22.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.021+0000 7fa91d6ee640 1 --2- 172.21.15.44:0/1353486968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa91806d470 0x7fa91814e600 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa908007f20 tx=0x7fa90800e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:22.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.021+0000 7fa915ffb640 1 -- 172.21.15.44:0/1353486968 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908014620 con 0x7fa91806d470 2024-04-22T06:31:22.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.021+0000 7fa915ffb640 1 -- 172.21.15.44:0/1353486968 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa908019070 con 0x7fa91806d470 2024-04-22T06:31:22.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.021+0000 7fa91f952640 1 -- 172.21.15.44:0/1353486968 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9180ccc10 con 0x7fa91806d470 2024-04-22T06:31:22.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.021+0000 7fa915ffb640 1 -- 172.21.15.44:0/1353486968 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908014a00 con 0x7fa91806d470 2024-04-22T06:31:22.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.021+0000 7fa91f952640 1 -- 172.21.15.44:0/1353486968 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa918156d70 con 0x7fa91806d470 2024-04-22T06:31:22.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.022+0000 7fa91f952640 1 -- 172.21.15.44:0/1353486968 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa918068cc0 con 0x7fa91806d470 2024-04-22T06:31:22.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.023+0000 7fa915ffb640 1 -- 172.21.15.44:0/1353486968 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa908014ba0 con 0x7fa91806d470 2024-04-22T06:31:22.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.023+0000 7fa915ffb640 1 --2- 172.21.15.44:0/1353486968 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fa90406c920 0x7fa90406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:22.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.023+0000 7fa915ffb640 1 -- 172.21.15.44:0/1353486968 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa90808c6e0 con 0x7fa91806d470 2024-04-22T06:31:22.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.023+0000 7fa91ceed640 1 --2- 172.21.15.44:0/1353486968 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fa90406c920 0x7fa90406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:22.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.024+0000 7fa91ceed640 1 --2- 172.21.15.44:0/1353486968 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fa90406c920 0x7fa90406ede0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa90c036000 tx=0x7fa90c008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:22.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.027+0000 7fa915ffb640 1 -- 172.21.15.44:0/1353486968 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa90805a7d0 con 0x7fa91806d470 2024-04-22T06:31:22.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.213+0000 7fa91f952640 1 -- 172.21.15.44:0/1353486968 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa9180ce130 con 0x7fa91806d470 2024-04-22T06:31:22.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.213+0000 7fa915ffb640 1 -- 172.21.15.44:0/1353486968 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7fa90805a0d0 con 0x7fa91806d470 2024-04-22T06:31:22.212 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-04-22T06:31:22.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.215+0000 7fa91f952640 1 -- 172.21.15.44:0/1353486968 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fa90406c920 msgr2=0x7fa90406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:22.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.215+0000 7fa91f952640 1 --2- 172.21.15.44:0/1353486968 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fa90406c920 0x7fa90406ede0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa90c036000 tx=0x7fa90c008030 comp rx=0 tx=0).stop 2024-04-22T06:31:22.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.215+0000 7fa91f952640 1 -- 172.21.15.44:0/1353486968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa91806d470 msgr2=0x7fa91814e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:22.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.215+0000 7fa91f952640 1 --2- 172.21.15.44:0/1353486968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa91806d470 0x7fa91814e600 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa908007f20 tx=0x7fa90800e570 comp rx=0 tx=0).stop 2024-04-22T06:31:22.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.215+0000 7fa91f952640 1 -- 172.21.15.44:0/1353486968 shutdown_connections 2024-04-22T06:31:22.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.215+0000 7fa91f952640 1 --2- 172.21.15.44:0/1353486968 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fa90406c920 0x7fa90406ede0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:22.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.215+0000 7fa91f952640 1 --2- 172.21.15.44:0/1353486968 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa9181552e0 0x7fa918157d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:22.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.215+0000 7fa91f952640 1 --2- 172.21.15.44:0/1353486968 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa918154f00 0x7fa918157850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:22.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.215+0000 7fa91f952640 1 --2- 172.21.15.44:0/1353486968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa91806d470 0x7fa91814e600 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:22.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.215+0000 7fa91f952640 1 -- 172.21.15.44:0/1353486968 >> 172.21.15.44:0/1353486968 conn(0x7fa9180733c0 msgr2=0x7fa91815cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:22.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.215+0000 7fa91f952640 1 -- 172.21.15.44:0/1353486968 shutdown_connections 2024-04-22T06:31:22.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.215+0000 7fa91f952640 1 -- 172.21.15.44:0/1353486968 wait complete. 2024-04-22T06:31:22.225 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T06:31:22.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.330+0000 7fe69d1ef640 1 Processor -- start 2024-04-22T06:31:22.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.330+0000 7fe69d1ef640 1 -- start start 2024-04-22T06:31:22.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.331+0000 7fe69d1ef640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe69815f050 0x7fe698073230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:22.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.331+0000 7fe69d1ef640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6980ccde0 con 0x7fe698155460 2024-04-22T06:31:22.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.331+0000 7fe69d1ef640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6980cd0a0 con 0x7fe698073770 2024-04-22T06:31:22.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.331+0000 7fe69d1ef640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fe698155840 con 0x7fe69815f050 2024-04-22T06:31:22.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.331+0000 7fe696d9d640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fe698155460 0x7fe6980cc6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:36624/0 (socket says 172.21.15.44:36624) 2024-04-22T06:31:22.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.331+0000 7fe696d9d640 1 -- 172.21.15.44:0/3569575648 learned_addr learned my addr 172.21.15.44:0/3569575648 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:22.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.331+0000 7fe69659c640 1 --2- 172.21.15.44:0/3569575648 >> v2:172.21.15.70:3300/0 conn(0x7fe69815f050 0x7fe698073230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:22.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.331+0000 7fe695d9b640 1 -- 172.21.15.44:0/3569575648 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 795839846 0 0) 0x7fe6980ccde0 con 0x7fe698155460 2024-04-22T06:31:22.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.331+0000 7fe695d9b640 1 -- 172.21.15.44:0/3569575648 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe684003520 con 0x7fe698155460 2024-04-22T06:31:22.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.332+0000 7fe69659c640 1 -- 172.21.15.44:0/3569575648 >> v1:172.21.15.70:6789/0 conn(0x7fe698073770 legacy=0x7fe6980c5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:22.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.332+0000 7fe69659c640 1 -- 172.21.15.44:0/3569575648 >> v1:172.21.15.44:6789/0 conn(0x7fe698155460 legacy=0x7fe6980cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:22.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.332+0000 7fe69659c640 1 -- 172.21.15.44:0/3569575648 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6980d0910 con 0x7fe69815f050 2024-04-22T06:31:22.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.332+0000 7fe69659c640 1 --2- 172.21.15.44:0/3569575648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe69815f050 0x7fe698073230 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe6800097a0 tx=0x7fe68002b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46365ceb1d446b8c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:22.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.333+0000 7fe695d9b640 1 -- 172.21.15.44:0/3569575648 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe680034070 con 0x7fe69815f050 2024-04-22T06:31:22.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.333+0000 7fe695d9b640 1 -- 172.21.15.44:0/3569575648 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe680030610 con 0x7fe69815f050 2024-04-22T06:31:22.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.333+0000 7fe695d9b640 1 -- 172.21.15.44:0/3569575648 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6800309c0 con 0x7fe69815f050 2024-04-22T06:31:22.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.333+0000 7fe69d1ef640 1 -- 172.21.15.44:0/3569575648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe69815f050 msgr2=0x7fe698073230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:22.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.333+0000 7fe69d1ef640 1 --2- 172.21.15.44:0/3569575648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe69815f050 0x7fe698073230 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe6800097a0 tx=0x7fe68002b960 comp rx=0 tx=0).stop 2024-04-22T06:31:22.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.333+0000 7fe69d1ef640 1 -- 172.21.15.44:0/3569575648 shutdown_connections 2024-04-22T06:31:22.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.333+0000 7fe69d1ef640 1 --2- 172.21.15.44:0/3569575648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe69815f050 0x7fe698073230 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:22.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.333+0000 7fe69d1ef640 1 -- 172.21.15.44:0/3569575648 >> 172.21.15.44:0/3569575648 conn(0x7fe69806cca0 msgr2=0x7fe69806d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:22.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.333+0000 7fe69d1ef640 1 -- 172.21.15.44:0/3569575648 shutdown_connections 2024-04-22T06:31:22.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.333+0000 7fe69d1ef640 1 -- 172.21.15.44:0/3569575648 wait complete. 2024-04-22T06:31:22.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.335+0000 7fe69d1ef640 1 Processor -- start 2024-04-22T06:31:22.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.335+0000 7fe69d1ef640 1 -- start start 2024-04-22T06:31:22.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.335+0000 7fe69d1ef640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe698073770 0x7fe698158ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:22.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.335+0000 7fe69d1ef640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe698155460 0x7fe6980c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:22.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.335+0000 7fe69d1ef640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe69815f050 0x7fe69815b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:22.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.335+0000 7fe69d1ef640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe6980d0ed0 con 0x7fe69815f050 2024-04-22T06:31:22.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.335+0000 7fe69d1ef640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe6980d1520 con 0x7fe698155460 2024-04-22T06:31:22.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.335+0000 7fe69d1ef640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe6980d11b0 con 0x7fe698073770 2024-04-22T06:31:22.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.335+0000 7fe69759e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe69815f050 0x7fe69815b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:22.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.336+0000 7fe69759e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe69815f050 0x7fe69815b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34174/0 (socket says 172.21.15.44:34174) 2024-04-22T06:31:22.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.336+0000 7fe69759e640 1 -- 172.21.15.44:0/2780832359 learned_addr learned my addr 172.21.15.44:0/2780832359 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:22.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.336+0000 7fe69659c640 1 --2- 172.21.15.44:0/2780832359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe698155460 0x7fe6980c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:22.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.336+0000 7fe69759e640 1 -- 172.21.15.44:0/2780832359 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe698073770 msgr2=0x7fe698158ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:22.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.336+0000 7fe696d9d640 1 --2- 172.21.15.44:0/2780832359 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe698073770 0x7fe698158ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:22.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.336+0000 7fe69759e640 1 --2- 172.21.15.44:0/2780832359 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe698073770 0x7fe698158ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:22.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.336+0000 7fe69759e640 1 -- 172.21.15.44:0/2780832359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe698155460 msgr2=0x7fe6980c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:22.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.336+0000 7fe69759e640 1 --2- 172.21.15.44:0/2780832359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe698155460 0x7fe6980c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:22.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.336+0000 7fe69759e640 1 -- 172.21.15.44:0/2780832359 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe69815a400 con 0x7fe69815f050 2024-04-22T06:31:22.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.336+0000 7fe69659c640 1 --2- 172.21.15.44:0/2780832359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe698155460 0x7fe6980c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:31:22.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.336+0000 7fe69759e640 1 --2- 172.21.15.44:0/2780832359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe69815f050 0x7fe69815b7e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe68800c7d0 tx=0x7fe68800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:22.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.336+0000 7fe68f7fe640 1 -- 172.21.15.44:0/2780832359 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe688003b70 con 0x7fe69815f050 2024-04-22T06:31:22.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.336+0000 7fe68f7fe640 1 -- 172.21.15.44:0/2780832359 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe688003d30 con 0x7fe69815f050 2024-04-22T06:31:22.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.336+0000 7fe69d1ef640 1 -- 172.21.15.44:0/2780832359 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6980c7ed0 con 0x7fe69815f050 2024-04-22T06:31:22.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.336+0000 7fe68f7fe640 1 -- 172.21.15.44:0/2780832359 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68800aa90 con 0x7fe69815f050 2024-04-22T06:31:22.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.336+0000 7fe69d1ef640 1 -- 172.21.15.44:0/2780832359 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe69815b2c0 con 0x7fe69815f050 2024-04-22T06:31:22.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.337+0000 7fe69d1ef640 1 -- 172.21.15.44:0/2780832359 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe698068cc0 con 0x7fe69815f050 2024-04-22T06:31:22.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.338+0000 7fe68f7fe640 1 -- 172.21.15.44:0/2780832359 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe68800ac30 con 0x7fe69815f050 2024-04-22T06:31:22.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.338+0000 7fe68f7fe640 1 --2- 172.21.15.44:0/2780832359 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fe67806c920 0x7fe67806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:22.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.339+0000 7fe68f7fe640 1 -- 172.21.15.44:0/2780832359 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe68808bd60 con 0x7fe69815f050 2024-04-22T06:31:22.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.339+0000 7fe696d9d640 1 --2- 172.21.15.44:0/2780832359 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fe67806c920 0x7fe67806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:22.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.339+0000 7fe696d9d640 1 --2- 172.21.15.44:0/2780832359 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fe67806c920 0x7fe67806ede0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe6900011e0 tx=0x7fe690009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:22.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.343+0000 7fe68f7fe640 1 -- 172.21.15.44:0/2780832359 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe688059ee0 con 0x7fe69815f050 2024-04-22T06:31:22.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.530+0000 7fe69d1ef640 1 -- 172.21.15.44:0/2780832359 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe69814f4e0 con 0x7fe69815f050 2024-04-22T06:31:22.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.530+0000 7fe68f7fe640 1 -- 172.21.15.44:0/2780832359 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7fe6880597e0 con 0x7fe69815f050 2024-04-22T06:31:22.529 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-04-22T06:31:22.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.532+0000 7fe69d1ef640 1 -- 172.21.15.44:0/2780832359 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fe67806c920 msgr2=0x7fe67806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:22.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.532+0000 7fe69d1ef640 1 --2- 172.21.15.44:0/2780832359 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fe67806c920 0x7fe67806ede0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe6900011e0 tx=0x7fe690009040 comp rx=0 tx=0).stop 2024-04-22T06:31:22.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.532+0000 7fe69d1ef640 1 -- 172.21.15.44:0/2780832359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe69815f050 msgr2=0x7fe69815b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:22.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.532+0000 7fe69d1ef640 1 --2- 172.21.15.44:0/2780832359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe69815f050 0x7fe69815b7e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe68800c7d0 tx=0x7fe68800cc90 comp rx=0 tx=0).stop 2024-04-22T06:31:22.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.532+0000 7fe69d1ef640 1 -- 172.21.15.44:0/2780832359 shutdown_connections 2024-04-22T06:31:22.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.532+0000 7fe69d1ef640 1 --2- 172.21.15.44:0/2780832359 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fe67806c920 0x7fe67806ede0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:22.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.532+0000 7fe69d1ef640 1 --2- 172.21.15.44:0/2780832359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe69815f050 0x7fe69815b7e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:22.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.532+0000 7fe69d1ef640 1 --2- 172.21.15.44:0/2780832359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe698155460 0x7fe6980c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:22.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.532+0000 7fe69d1ef640 1 --2- 172.21.15.44:0/2780832359 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe698073770 0x7fe698158ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:22.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.532+0000 7fe69d1ef640 1 -- 172.21.15.44:0/2780832359 >> 172.21.15.44:0/2780832359 conn(0x7fe69806cca0 msgr2=0x7fe69806d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:22.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.532+0000 7fe69d1ef640 1 -- 172.21.15.44:0/2780832359 shutdown_connections 2024-04-22T06:31:22.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.532+0000 7fe69d1ef640 1 -- 172.21.15.44:0/2780832359 wait complete. 2024-04-22T06:31:22.542 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-22T06:31:22.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.647+0000 7f548ff38640 1 Processor -- start 2024-04-22T06:31:22.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.647+0000 7f548ff38640 1 -- start start 2024-04-22T06:31:22.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.648+0000 7f548ff38640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f548806d470 0x7f54880cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:22.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.648+0000 7f548ff38640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f548815ccf0 con 0x7f5488154f00 2024-04-22T06:31:22.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.648+0000 7f548ff38640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54880c5b30 con 0x7f54881552e0 2024-04-22T06:31:22.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.648+0000 7f548ff38640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f54880c6740 con 0x7f548806d470 2024-04-22T06:31:22.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.648+0000 7f548dcd4640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f5488154f00 0x7f548815c5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:36638/0 (socket says 172.21.15.44:36638) 2024-04-22T06:31:22.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.648+0000 7f548dcd4640 1 -- 172.21.15.44:0/4029792903 learned_addr learned my addr 172.21.15.44:0/4029792903 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:22.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.648+0000 7f548e4d5640 1 --2- 172.21.15.44:0/4029792903 >> v2:172.21.15.70:3300/0 conn(0x7f548806d470 0x7f54880cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:22.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.648+0000 7f548ccd2640 1 -- 172.21.15.44:0/4029792903 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 59431607 0 0) 0x7f548815ccf0 con 0x7f5488154f00 2024-04-22T06:31:22.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.648+0000 7f548ccd2640 1 -- 172.21.15.44:0/4029792903 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5470003520 con 0x7f5488154f00 2024-04-22T06:31:22.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.648+0000 7f548ccd2640 1 -- 172.21.15.44:0/4029792903 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4099467597 0 0) 0x7f54880c5b30 con 0x7f54881552e0 2024-04-22T06:31:22.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.648+0000 7f548ccd2640 1 -- 172.21.15.44:0/4029792903 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f548815ccf0 con 0x7f54881552e0 2024-04-22T06:31:22.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.648+0000 7f548ccd2640 1 -- 172.21.15.44:0/4029792903 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2457696297 0 0) 0x7f5470003520 con 0x7f5488154f00 2024-04-22T06:31:22.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.648+0000 7f548e4d5640 1 -- 172.21.15.44:0/4029792903 >> v1:172.21.15.70:6789/0 conn(0x7f54881552e0 legacy=0x7f54880d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:22.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.648+0000 7f548e4d5640 1 -- 172.21.15.44:0/4029792903 >> v1:172.21.15.44:6789/0 conn(0x7f5488154f00 legacy=0x7f548815c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:22.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.648+0000 7f548e4d5640 1 -- 172.21.15.44:0/4029792903 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54880c5e10 con 0x7f548806d470 2024-04-22T06:31:22.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.649+0000 7f548e4d5640 1 --2- 172.21.15.44:0/4029792903 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f548806d470 0x7f54880cb430 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5478009d90 tx=0x7f547802bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14de178aae3f463d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:22.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.649+0000 7f548ccd2640 1 -- 172.21.15.44:0/4029792903 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5478039070 con 0x7f548806d470 2024-04-22T06:31:22.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.649+0000 7f548ccd2640 1 -- 172.21.15.44:0/4029792903 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f547802c8b0 con 0x7f548806d470 2024-04-22T06:31:22.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.649+0000 7f548ccd2640 1 -- 172.21.15.44:0/4029792903 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5480001670 con 0x7f548806d470 2024-04-22T06:31:22.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.650+0000 7f548ff38640 1 -- 172.21.15.44:0/4029792903 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f548806d470 msgr2=0x7f54880cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:22.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.650+0000 7f548ff38640 1 --2- 172.21.15.44:0/4029792903 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f548806d470 0x7f54880cb430 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5478009d90 tx=0x7f547802bd80 comp rx=0 tx=0).stop 2024-04-22T06:31:22.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.650+0000 7f548ff38640 1 -- 172.21.15.44:0/4029792903 shutdown_connections 2024-04-22T06:31:22.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.650+0000 7f548ff38640 1 --2- 172.21.15.44:0/4029792903 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f548806d470 0x7f54880cb430 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:22.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.650+0000 7f548ff38640 1 -- 172.21.15.44:0/4029792903 >> 172.21.15.44:0/4029792903 conn(0x7f54880733c0 msgr2=0x7f54880737d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:22.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.650+0000 7f548ff38640 1 -- 172.21.15.44:0/4029792903 shutdown_connections 2024-04-22T06:31:22.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.650+0000 7f548ff38640 1 -- 172.21.15.44:0/4029792903 wait complete. 2024-04-22T06:31:22.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.651+0000 7f548ff38640 1 Processor -- start 2024-04-22T06:31:22.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.651+0000 7f548ff38640 1 -- start start 2024-04-22T06:31:22.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.652+0000 7f548ff38640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f548806d470 0x7f54880b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:22.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.652+0000 7f548ff38640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5488154f00 0x7f548815d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:22.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.652+0000 7f548ff38640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54881552e0 0x7f54881f12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:22.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.652+0000 7f548ff38640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f54880c6bd0 con 0x7f54881552e0 2024-04-22T06:31:22.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.652+0000 7f548ff38640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f54880c63d0 con 0x7f5488154f00 2024-04-22T06:31:22.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.652+0000 7f548ff38640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f54880c6d50 con 0x7f548806d470 2024-04-22T06:31:22.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.652+0000 7f548e4d5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54881552e0 0x7f54881f12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:22.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.652+0000 7f548e4d5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54881552e0 0x7f54881f12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34180/0 (socket says 172.21.15.44:34180) 2024-04-22T06:31:22.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.652+0000 7f548e4d5640 1 -- 172.21.15.44:0/1157310597 learned_addr learned my addr 172.21.15.44:0/1157310597 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:22.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.652+0000 7f548d4d3640 1 --2- 172.21.15.44:0/1157310597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5488154f00 0x7f548815d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:22.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.652+0000 7f548dcd4640 1 --2- 172.21.15.44:0/1157310597 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f548806d470 0x7f54880b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:22.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.652+0000 7f548e4d5640 1 -- 172.21.15.44:0/1157310597 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f548806d470 msgr2=0x7f54880b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:22.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.652+0000 7f548e4d5640 1 --2- 172.21.15.44:0/1157310597 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f548806d470 0x7f54880b3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:22.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.652+0000 7f548e4d5640 1 -- 172.21.15.44:0/1157310597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5488154f00 msgr2=0x7f548815d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:22.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.652+0000 7f548e4d5640 1 --2- 172.21.15.44:0/1157310597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5488154f00 0x7f548815d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:22.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.652+0000 7f548e4d5640 1 -- 172.21.15.44:0/1157310597 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54880cbcd0 con 0x7f54881552e0 2024-04-22T06:31:22.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.652+0000 7f548e4d5640 1 --2- 172.21.15.44:0/1157310597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54881552e0 0x7f54881f12e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5478037040 tx=0x7f5478004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:22.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.652+0000 7f54867fc640 1 -- 172.21.15.44:0/1157310597 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5478039050 con 0x7f54881552e0 2024-04-22T06:31:22.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.652+0000 7f54867fc640 1 -- 172.21.15.44:0/1157310597 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f547803a040 con 0x7f54881552e0 2024-04-22T06:31:22.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.652+0000 7f548ff38640 1 -- 172.21.15.44:0/1157310597 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f548815a960 con 0x7f54881552e0 2024-04-22T06:31:22.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.652+0000 7f54867fc640 1 -- 172.21.15.44:0/1157310597 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5478009450 con 0x7f54881552e0 2024-04-22T06:31:22.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.653+0000 7f548ff38640 1 -- 172.21.15.44:0/1157310597 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54880cc250 con 0x7f54881552e0 2024-04-22T06:31:22.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.653+0000 7f548ff38640 1 -- 172.21.15.44:0/1157310597 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5488068cc0 con 0x7f54881552e0 2024-04-22T06:31:22.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.654+0000 7f54867fc640 1 -- 172.21.15.44:0/1157310597 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5478047020 con 0x7f54881552e0 2024-04-22T06:31:22.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.654+0000 7f54867fc640 1 --2- 172.21.15.44:0/1157310597 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f547406c8f0 0x7f547406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:22.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.654+0000 7f54867fc640 1 -- 172.21.15.44:0/1157310597 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f54780adbc0 con 0x7f54881552e0 2024-04-22T06:31:22.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.654+0000 7f548dcd4640 1 --2- 172.21.15.44:0/1157310597 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f547406c8f0 0x7f547406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:22.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.655+0000 7f548dcd4640 1 --2- 172.21.15.44:0/1157310597 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f547406c8f0 0x7f547406edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f54800017e0 tx=0x7f5480009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:22.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.659+0000 7f54867fc640 1 -- 172.21.15.44:0/1157310597 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f547804e020 con 0x7f54881552e0 2024-04-22T06:31:22.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.842+0000 7f548ff38640 1 -- 172.21.15.44:0/1157310597 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f54880c7490 con 0x7f54881552e0 2024-04-22T06:31:22.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.843+0000 7f54867fc640 1 -- 172.21.15.44:0/1157310597 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f5478032340 con 0x7f54881552e0 2024-04-22T06:31:22.842 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-04-22T06:31:22.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.845+0000 7f548ff38640 1 -- 172.21.15.44:0/1157310597 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f547406c8f0 msgr2=0x7f547406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:22.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.845+0000 7f548ff38640 1 --2- 172.21.15.44:0/1157310597 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f547406c8f0 0x7f547406edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f54800017e0 tx=0x7f5480009040 comp rx=0 tx=0).stop 2024-04-22T06:31:22.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.845+0000 7f548ff38640 1 -- 172.21.15.44:0/1157310597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54881552e0 msgr2=0x7f54881f12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:22.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.845+0000 7f548ff38640 1 --2- 172.21.15.44:0/1157310597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54881552e0 0x7f54881f12e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5478037040 tx=0x7f5478004a40 comp rx=0 tx=0).stop 2024-04-22T06:31:22.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.845+0000 7f548ff38640 1 -- 172.21.15.44:0/1157310597 shutdown_connections 2024-04-22T06:31:22.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.845+0000 7f548ff38640 1 --2- 172.21.15.44:0/1157310597 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f547406c8f0 0x7f547406edb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:22.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.845+0000 7f548ff38640 1 --2- 172.21.15.44:0/1157310597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54881552e0 0x7f54881f12e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:22.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.845+0000 7f548ff38640 1 --2- 172.21.15.44:0/1157310597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5488154f00 0x7f548815d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:22.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.845+0000 7f548ff38640 1 --2- 172.21.15.44:0/1157310597 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f548806d470 0x7f54880b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:22.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.845+0000 7f548ff38640 1 -- 172.21.15.44:0/1157310597 >> 172.21.15.44:0/1157310597 conn(0x7f54880733c0 msgr2=0x7f54881534a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:22.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.845+0000 7f548ff38640 1 -- 172.21.15.44:0/1157310597 shutdown_connections 2024-04-22T06:31:22.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.845+0000 7f548ff38640 1 -- 172.21.15.44:0/1157310597 wait complete. 2024-04-22T06:31:22.855 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-22T06:31:22.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.957+0000 7f0814c81640 1 Processor -- start 2024-04-22T06:31:22.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.957+0000 7f0814c81640 1 -- start start 2024-04-22T06:31:22.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.958+0000 7f0814c81640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f08100cb6c0 0x7f08100c8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:22.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.958+0000 7f0814c81640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0810158fd0 con 0x7f08100cbaa0 2024-04-22T06:31:22.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.958+0000 7f0814c81640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08100c5e40 con 0x7f08100c5890 2024-04-22T06:31:22.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.958+0000 7f0814c81640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f08100c8190 con 0x7f08100cb6c0 2024-04-22T06:31:22.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.958+0000 7f080e59c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f08100cb6c0 0x7f08100c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:22.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.958+0000 7f080ed9d640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f08100cbaa0 0x7f08100c93c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:36654/0 (socket says 172.21.15.44:36654) 2024-04-22T06:31:22.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.958+0000 7f080e59c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f08100cb6c0 0x7f08100c8620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34186/0 (socket says 172.21.15.44:34186) 2024-04-22T06:31:22.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.958+0000 7f080ed9d640 1 -- 172.21.15.44:0/2433410829 learned_addr learned my addr 172.21.15.44:0/2433410829 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:22.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.959+0000 7f080e59c640 1 -- 172.21.15.44:0/2433410829 >> v1:172.21.15.70:6789/0 conn(0x7f08100c5890 legacy=0x7f081014f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:22.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.959+0000 7f080e59c640 1 -- 172.21.15.44:0/2433410829 >> v1:172.21.15.44:6789/0 conn(0x7f08100cbaa0 legacy=0x7f08100c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:22.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.959+0000 7f080e59c640 1 -- 172.21.15.44:0/2433410829 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08100c6120 con 0x7f08100cb6c0 2024-04-22T06:31:22.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.959+0000 7f080e59c640 1 --2- 172.21.15.44:0/2433410829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08100cb6c0 0x7f08100c8620 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f08040097a0 tx=0x7f080402b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ff3be4176ff51 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:22.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.960+0000 7f080d59a640 1 -- 172.21.15.44:0/2433410829 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080402f810 con 0x7f08100cb6c0 2024-04-22T06:31:22.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.960+0000 7f080d59a640 1 -- 172.21.15.44:0/2433410829 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f080402f9b0 con 0x7f08100cb6c0 2024-04-22T06:31:22.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.960+0000 7f080d59a640 1 -- 172.21.15.44:0/2433410829 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080402fd80 con 0x7f08100cb6c0 2024-04-22T06:31:22.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.960+0000 7f0814c81640 1 -- 172.21.15.44:0/2433410829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08100cb6c0 msgr2=0x7f08100c8620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:22.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.960+0000 7f0814c81640 1 --2- 172.21.15.44:0/2433410829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08100cb6c0 0x7f08100c8620 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f08040097a0 tx=0x7f080402b9f0 comp rx=0 tx=0).stop 2024-04-22T06:31:22.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.960+0000 7f0814c81640 1 -- 172.21.15.44:0/2433410829 shutdown_connections 2024-04-22T06:31:22.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.960+0000 7f0814c81640 1 --2- 172.21.15.44:0/2433410829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08100cb6c0 0x7f08100c8620 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:22.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.960+0000 7f0814c81640 1 -- 172.21.15.44:0/2433410829 >> 172.21.15.44:0/2433410829 conn(0x7f081006fc40 msgr2=0x7f0810074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:22.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.960+0000 7f0814c81640 1 -- 172.21.15.44:0/2433410829 shutdown_connections 2024-04-22T06:31:22.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.960+0000 7f0814c81640 1 -- 172.21.15.44:0/2433410829 wait complete. 2024-04-22T06:31:22.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.962+0000 7f0814c81640 1 Processor -- start 2024-04-22T06:31:22.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.962+0000 7f0814c81640 1 -- start start 2024-04-22T06:31:22.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.962+0000 7f0814c81640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08100c5890 0x7f0810147410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:22.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.962+0000 7f0814c81640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08100cb6c0 0x7f0810148a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:22.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.962+0000 7f0814c81640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08100cbaa0 0x7f081015bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:22.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.962+0000 7f0814c81640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f08100c69c0 con 0x7f08100cbaa0 2024-04-22T06:31:22.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.962+0000 7f0814c81640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f08100c66e0 con 0x7f08100c5890 2024-04-22T06:31:22.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.962+0000 7f0814c81640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f081014fc90 con 0x7f08100cb6c0 2024-04-22T06:31:22.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.962+0000 7f080ed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08100cbaa0 0x7f081015bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:22.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.962+0000 7f080ed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08100cbaa0 0x7f081015bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34202/0 (socket says 172.21.15.44:34202) 2024-04-22T06:31:22.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.962+0000 7f080ed9d640 1 -- 172.21.15.44:0/55193896 learned_addr learned my addr 172.21.15.44:0/55193896 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:22.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.963+0000 7f080dd9b640 1 --2- 172.21.15.44:0/55193896 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08100cb6c0 0x7f0810148a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:22.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.963+0000 7f080e59c640 1 --2- 172.21.15.44:0/55193896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08100c5890 0x7f0810147410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:22.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.963+0000 7f080ed9d640 1 -- 172.21.15.44:0/55193896 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08100cb6c0 msgr2=0x7f0810148a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:22.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.963+0000 7f080ed9d640 1 --2- 172.21.15.44:0/55193896 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08100cb6c0 0x7f0810148a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:22.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.963+0000 7f080ed9d640 1 -- 172.21.15.44:0/55193896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08100c5890 msgr2=0x7f0810147410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:22.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.963+0000 7f080ed9d640 1 --2- 172.21.15.44:0/55193896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08100c5890 0x7f0810147410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:22.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.963+0000 7f080ed9d640 1 -- 172.21.15.44:0/55193896 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08100cd500 con 0x7f08100cbaa0 2024-04-22T06:31:22.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.963+0000 7f080ed9d640 1 --2- 172.21.15.44:0/55193896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08100cbaa0 0x7f081015bed0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0808002230 tx=0x7f080800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:22.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.963+0000 7f07feffd640 1 -- 172.21.15.44:0/55193896 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0808017070 con 0x7f08100cbaa0 2024-04-22T06:31:22.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.963+0000 7f07feffd640 1 -- 172.21.15.44:0/55193896 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f080800abd0 con 0x7f08100cbaa0 2024-04-22T06:31:22.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.963+0000 7f0814c81640 1 -- 172.21.15.44:0/55193896 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08100cd7e0 con 0x7f08100cbaa0 2024-04-22T06:31:22.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.963+0000 7f07feffd640 1 -- 172.21.15.44:0/55193896 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0808012400 con 0x7f08100cbaa0 2024-04-22T06:31:22.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.963+0000 7f0814c81640 1 -- 172.21.15.44:0/55193896 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0810148280 con 0x7f08100cbaa0 2024-04-22T06:31:22.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.964+0000 7f0814c81640 1 -- 172.21.15.44:0/55193896 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0810069310 con 0x7f08100cbaa0 2024-04-22T06:31:22.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.964+0000 7f07feffd640 1 -- 172.21.15.44:0/55193896 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f080801a070 con 0x7f08100cbaa0 2024-04-22T06:31:22.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.965+0000 7f07feffd640 1 --2- 172.21.15.44:0/55193896 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f07f006c920 0x7f07f006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:22.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.965+0000 7f07feffd640 1 -- 172.21.15.44:0/55193896 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f080808ba10 con 0x7f08100cbaa0 2024-04-22T06:31:22.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.965+0000 7f080e59c640 1 --2- 172.21.15.44:0/55193896 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f07f006c920 0x7f07f006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:22.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.966+0000 7f080e59c640 1 --2- 172.21.15.44:0/55193896 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f07f006c920 0x7f07f006ede0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0804036000 tx=0x7f0804008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:22.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:22.969+0000 7f07feffd640 1 -- 172.21.15.44:0/55193896 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f08080594a0 con 0x7f08100cbaa0 2024-04-22T06:31:23.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:23.168+0000 7f0814c81640 1 -- 172.21.15.44:0/55193896 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f08100c9b50 con 0x7f08100cbaa0 2024-04-22T06:31:23.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:23.168+0000 7f07feffd640 1 -- 172.21.15.44:0/55193896 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7f080805d7a0 con 0x7f08100cbaa0 2024-04-22T06:31:23.167 INFO:teuthology.orchestra.run.smithi044.stderr:listed 0 entries 2024-04-22T06:31:23.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:23.170+0000 7f0814c81640 1 -- 172.21.15.44:0/55193896 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f07f006c920 msgr2=0x7f07f006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:23.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:23.170+0000 7f0814c81640 1 --2- 172.21.15.44:0/55193896 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f07f006c920 0x7f07f006ede0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0804036000 tx=0x7f0804008030 comp rx=0 tx=0).stop 2024-04-22T06:31:23.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:23.170+0000 7f0814c81640 1 -- 172.21.15.44:0/55193896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08100cbaa0 msgr2=0x7f081015bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:23.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:23.170+0000 7f0814c81640 1 --2- 172.21.15.44:0/55193896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08100cbaa0 0x7f081015bed0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0808002230 tx=0x7f080800cf70 comp rx=0 tx=0).stop 2024-04-22T06:31:23.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:23.170+0000 7f0814c81640 1 -- 172.21.15.44:0/55193896 shutdown_connections 2024-04-22T06:31:23.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:23.170+0000 7f0814c81640 1 --2- 172.21.15.44:0/55193896 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f07f006c920 0x7f07f006ede0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:23.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:23.170+0000 7f0814c81640 1 --2- 172.21.15.44:0/55193896 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08100cb6c0 0x7f0810148a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:23.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:23.170+0000 7f0814c81640 1 --2- 172.21.15.44:0/55193896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08100cbaa0 0x7f081015bed0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:23.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:23.170+0000 7f0814c81640 1 --2- 172.21.15.44:0/55193896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08100c5890 0x7f0810147410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:23.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:23.170+0000 7f0814c81640 1 -- 172.21.15.44:0/55193896 >> 172.21.15.44:0/55193896 conn(0x7f081006fc40 msgr2=0x7f0810159310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:23.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:23.170+0000 7f0814c81640 1 -- 172.21.15.44:0/55193896 shutdown_connections 2024-04-22T06:31:23.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:23.170+0000 7f0814c81640 1 -- 172.21.15.44:0/55193896 wait complete. 2024-04-22T06:31:23.180 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi044.front.sepia.ceph.com... 2024-04-22T06:31:23.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T06:31:23.180 DEBUG:teuthology.orchestra.run.smithi044:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-22T06:31:23.203 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi044.stderr:ceph-fuse[41460]: fuse finished with error 0 and tester_r 0 2024-04-22T06:31:23.205 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T06:31:29.306 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-04-22T06:31:29.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T06:31:29.307 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-22T06:31:29.329 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T06:31:29.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.477+0000 7f8a61515640 1 Processor -- start 2024-04-22T06:31:29.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.477+0000 7f8a61515640 1 -- start start 2024-04-22T06:31:29.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.478+0000 7f8a61515640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8a5c1552a0 0x7f8a5c15bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:29.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.478+0000 7f8a61515640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a5c06c6e0 con 0x7f8a5c155680 2024-04-22T06:31:29.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.478+0000 7f8a61515640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a5c06c9c0 con 0x7f8a5c0cc4e0 2024-04-22T06:31:29.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.478+0000 7f8a61515640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f8a5c0cb990 con 0x7f8a5c1552a0 2024-04-22T06:31:29.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.478+0000 7f8a5a7fc640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f8a5c155680 0x7f8a5c0cbdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:36670/0 (socket says 172.21.15.44:36670) 2024-04-22T06:31:29.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.478+0000 7f8a5a7fc640 1 -- 172.21.15.44:0/3589666979 learned_addr learned my addr 172.21.15.44:0/3589666979 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:29.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.478+0000 7f8a5affd640 1 --2- 172.21.15.44:0/3589666979 >> v2:172.21.15.70:3300/0 conn(0x7f8a5c1552a0 0x7f8a5c15bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:29.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.478+0000 7f8a59ffb640 1 -- 172.21.15.44:0/3589666979 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2686859944 0 0) 0x7f8a5c06c6e0 con 0x7f8a5c155680 2024-04-22T06:31:29.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.478+0000 7f8a59ffb640 1 -- 172.21.15.44:0/3589666979 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a44003520 con 0x7f8a5c155680 2024-04-22T06:31:29.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.478+0000 7f8a59ffb640 1 -- 172.21.15.44:0/3589666979 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3553647847 0 0) 0x7f8a44003520 con 0x7f8a5c155680 2024-04-22T06:31:29.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.478+0000 7f8a59ffb640 1 -- 172.21.15.44:0/3589666979 >> v2:172.21.15.70:3300/0 conn(0x7f8a5c1552a0 msgr2=0x7f8a5c15bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:29.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.478+0000 7f8a59ffb640 1 --2- 172.21.15.44:0/3589666979 >> v2:172.21.15.70:3300/0 conn(0x7f8a5c1552a0 0x7f8a5c15bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:29.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.478+0000 7f8a59ffb640 1 -- 172.21.15.44:0/3589666979 >> v1:172.21.15.70:6789/0 conn(0x7f8a5c0cc4e0 legacy=0x7f8a5c0ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:29.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.478+0000 7f8a59ffb640 1 -- 172.21.15.44:0/3589666979 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a5c06cca0 con 0x7f8a5c155680 2024-04-22T06:31:29.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.478+0000 7f8a59ffb640 1 -- 172.21.15.44:0/3589666979 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1237339257 0 0) 0x7f8a50001670 con 0x7f8a5c155680 2024-04-22T06:31:29.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.478+0000 7f8a59ffb640 1 -- 172.21.15.44:0/3589666979 >> v1:172.21.15.44:6789/0 conn(0x7f8a5c155680 legacy=0x7f8a5c0cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:29.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.478+0000 7f8a59ffb640 1 --2- 172.21.15.44:0/3589666979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a440039b0 0x7f8a44003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:29.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.478+0000 7f8a59ffb640 1 --2- 172.21.15.44:0/3589666979 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8a44004800 0x7f8a44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:29.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.478+0000 7f8a59ffb640 1 --2- 172.21.15.44:0/3589666979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a440051c0 0x7f8a4400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:29.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.479+0000 7f8a59ffb640 1 -- 172.21.15.44:0/3589666979 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c0cb990 con 0x7f8a440051c0 2024-04-22T06:31:29.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.479+0000 7f8a59ffb640 1 -- 172.21.15.44:0/3589666979 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4400c070 con 0x7f8a440039b0 2024-04-22T06:31:29.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.479+0000 7f8a59ffb640 1 -- 172.21.15.44:0/3589666979 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8a4400c270 con 0x7f8a44004800 2024-04-22T06:31:29.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.479+0000 7f8a5affd640 1 --2- 172.21.15.44:0/3589666979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a440039b0 0x7f8a44003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:29.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.479+0000 7f8a5b7fe640 1 --2- 172.21.15.44:0/3589666979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a440051c0 0x7f8a4400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:29.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.479+0000 7f8a5a7fc640 1 --2- 172.21.15.44:0/3589666979 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8a44004800 0x7f8a44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:29.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.479+0000 7f8a5b7fe640 1 -- 172.21.15.44:0/3589666979 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8a44004800 msgr2=0x7f8a44004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:29.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.479+0000 7f8a5b7fe640 1 --2- 172.21.15.44:0/3589666979 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8a44004800 0x7f8a44004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:29.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.479+0000 7f8a5b7fe640 1 -- 172.21.15.44:0/3589666979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a440039b0 msgr2=0x7f8a44003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:29.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.479+0000 7f8a5b7fe640 1 --2- 172.21.15.44:0/3589666979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a440039b0 0x7f8a44003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:29.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.479+0000 7f8a5b7fe640 1 -- 172.21.15.44:0/3589666979 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a44003520 con 0x7f8a440051c0 2024-04-22T06:31:29.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.479+0000 7f8a5affd640 1 --2- 172.21.15.44:0/3589666979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a440039b0 0x7f8a44003df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:31:29.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.480+0000 7f8a5b7fe640 1 -- 172.21.15.44:0/3589666979 reap_dead start 2024-04-22T06:31:29.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.480+0000 7f8a5b7fe640 1 --2- 172.21.15.44:0/3589666979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a440051c0 0x7f8a4400b9c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8a54001e50 tx=0x7f8a44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c58e23c6d771c30 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:29.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.480+0000 7f8a59ffb640 1 -- 172.21.15.44:0/3589666979 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c009690 con 0x7f8a440051c0 2024-04-22T06:31:29.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.480+0000 7f8a59ffb640 1 -- 172.21.15.44:0/3589666979 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a54003bb0 con 0x7f8a440051c0 2024-04-22T06:31:29.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.480+0000 7f8a59ffb640 1 -- 172.21.15.44:0/3589666979 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a54001a70 con 0x7f8a440051c0 2024-04-22T06:31:29.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.480+0000 7f8a61515640 1 -- 172.21.15.44:0/3589666979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a440051c0 msgr2=0x7f8a4400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:29.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.480+0000 7f8a61515640 1 --2- 172.21.15.44:0/3589666979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a440051c0 0x7f8a4400b9c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8a54001e50 tx=0x7f8a44005830 comp rx=0 tx=0).stop 2024-04-22T06:31:29.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.480+0000 7f8a61515640 1 -- 172.21.15.44:0/3589666979 shutdown_connections 2024-04-22T06:31:29.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.480+0000 7f8a61515640 1 --2- 172.21.15.44:0/3589666979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a440051c0 0x7f8a4400b9c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:29.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.480+0000 7f8a61515640 1 -- 172.21.15.44:0/3589666979 >> 172.21.15.44:0/3589666979 conn(0x7f8a5c075820 msgr2=0x7f8a5c073000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:29.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.480+0000 7f8a61515640 1 -- 172.21.15.44:0/3589666979 shutdown_connections 2024-04-22T06:31:29.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.480+0000 7f8a61515640 1 -- 172.21.15.44:0/3589666979 wait complete. 2024-04-22T06:31:29.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.482+0000 7f8a61515640 1 Processor -- start 2024-04-22T06:31:29.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.482+0000 7f8a61515640 1 -- start start 2024-04-22T06:31:29.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.482+0000 7f8a61515640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8a440051c0 0x7f8a5c0d0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:29.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.482+0000 7f8a61515640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a5c0c7d00 0x7f8a5c1f3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:29.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.482+0000 7f8a61515640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a5c15c260 0x7f8a5c1f4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:29.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.482+0000 7f8a61515640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c06d540 con 0x7f8a5c15c260 2024-04-22T06:31:29.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.482+0000 7f8a61515640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c06d260 con 0x7f8a5c0c7d00 2024-04-22T06:31:29.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.482+0000 7f8a61515640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8a5c0b6690 con 0x7f8a440051c0 2024-04-22T06:31:29.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.483+0000 7f8a5b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a5c15c260 0x7f8a5c1f4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:29.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.483+0000 7f8a5b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a5c15c260 0x7f8a5c1f4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34224/0 (socket says 172.21.15.44:34224) 2024-04-22T06:31:29.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.483+0000 7f8a5b7fe640 1 -- 172.21.15.44:0/2615088088 learned_addr learned my addr 172.21.15.44:0/2615088088 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:29.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.483+0000 7f8a5affd640 1 --2- 172.21.15.44:0/2615088088 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8a440051c0 0x7f8a5c0d0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:29.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.483+0000 7f8a5a7fc640 1 --2- 172.21.15.44:0/2615088088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a5c0c7d00 0x7f8a5c1f3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:29.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.483+0000 7f8a5b7fe640 1 -- 172.21.15.44:0/2615088088 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8a440051c0 msgr2=0x7f8a5c0d0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:29.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.483+0000 7f8a5b7fe640 1 --2- 172.21.15.44:0/2615088088 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8a440051c0 0x7f8a5c0d0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:29.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.483+0000 7f8a5b7fe640 1 -- 172.21.15.44:0/2615088088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a5c0c7d00 msgr2=0x7f8a5c1f3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:29.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.483+0000 7f8a5b7fe640 1 --2- 172.21.15.44:0/2615088088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a5c0c7d00 0x7f8a5c1f3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:29.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.483+0000 7f8a5b7fe640 1 -- 172.21.15.44:0/2615088088 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a5c0c3fb0 con 0x7f8a5c15c260 2024-04-22T06:31:29.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.483+0000 7f8a5b7fe640 1 --2- 172.21.15.44:0/2615088088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a5c15c260 0x7f8a5c1f4510 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8a5402c060 tx=0x7f8a540014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:29.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.483+0000 7f8a437fe640 1 -- 172.21.15.44:0/2615088088 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a54007db0 con 0x7f8a5c15c260 2024-04-22T06:31:29.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.483+0000 7f8a437fe640 1 -- 172.21.15.44:0/2615088088 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a54006d60 con 0x7f8a5c15c260 2024-04-22T06:31:29.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.483+0000 7f8a61515640 1 -- 172.21.15.44:0/2615088088 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a5c0c5880 con 0x7f8a5c15c260 2024-04-22T06:31:29.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.483+0000 7f8a437fe640 1 -- 172.21.15.44:0/2615088088 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a54039680 con 0x7f8a5c15c260 2024-04-22T06:31:29.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.483+0000 7f8a61515640 1 -- 172.21.15.44:0/2615088088 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a5c1f4e50 con 0x7f8a5c15c260 2024-04-22T06:31:29.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.484+0000 7f8a61515640 1 -- 172.21.15.44:0/2615088088 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a5c068cc0 con 0x7f8a5c15c260 2024-04-22T06:31:29.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.485+0000 7f8a437fe640 1 -- 172.21.15.44:0/2615088088 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a54039820 con 0x7f8a5c15c260 2024-04-22T06:31:29.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.485+0000 7f8a437fe640 1 --2- 172.21.15.44:0/2615088088 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f8a4806c920 0x7f8a4806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:29.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.485+0000 7f8a437fe640 1 -- 172.21.15.44:0/2615088088 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8a540adce0 con 0x7f8a5c15c260 2024-04-22T06:31:29.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.485+0000 7f8a5affd640 1 --2- 172.21.15.44:0/2615088088 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f8a4806c920 0x7f8a4806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:29.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.486+0000 7f8a5affd640 1 --2- 172.21.15.44:0/2615088088 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f8a4806c920 0x7f8a4806ede0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c009b40 tx=0x7f8a4c009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:29.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.490+0000 7f8a437fe640 1 -- 172.21.15.44:0/2615088088 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8a54033030 con 0x7f8a5c15c260 2024-04-22T06:31:29.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.676+0000 7f8a61515640 1 -- 172.21.15.44:0/2615088088 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8a5c159010 con 0x7f8a5c15c260 2024-04-22T06:31:29.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.677+0000 7f8a437fe640 1 -- 172.21.15.44:0/2615088088 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18466 (secure 0 0 0) 0x7f8a5407be00 con 0x7f8a5c15c260 2024-04-22T06:31:29.676 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:31:29.676 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"fsid":"9b0a34ca-403d-4513-8a66-d3749ba559aa","created":"2024-04-22T06:30:42.334581+0000","modified":"2024-04-22T06:31:11.406256+0000","last_up_change":"2024-04-22T06:30:52.942257+0000","last_in_change":"2024-04-22T06:30:44.467727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T06:30:54.560256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T06:31:04.612387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T06:31:05.312124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-22T06:31:09.025550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b4a6692-53c9-472d-b2f3-1dd29a6679ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6817","nonce":2220268493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6819","nonce":2220268493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6823","nonce":2220268493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6821","nonce":2220268493}]},"public_addr":"172.21.15.44:6817/2220268493","cluster_addr":"172.21.15.44:6819/2220268493","heartbeat_back_addr":"172.21.15.44:6823/2220268493","heartbeat_front_addr":"172.21.15.44:6821/2220268493","state":["exists","up"]},{"osd":1,"uuid":"70a6370a-7c0a-4d64-a0e4-b2e726b4f291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6809","nonce":1278678977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6811","nonce":1278678977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6815","nonce":1278678977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6813","nonce":1278678977}]},"public_addr":"172.21.15.44:6809/1278678977","cluster_addr":"172.21.15.44:6811/1278678977","heartbeat_back_addr":"172.21.15.44:6815/1278678977","heartbeat_front_addr":"172.21.15.44:6813/1278678977","state":["exists","up"]},{"osd":2,"uuid":"f2e51a62-ca91-470d-bd5a-5710c61deee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6801","nonce":2544087903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6803","nonce":2544087903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6807","nonce":2544087903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6805","nonce":2544087903}]},"public_addr":"172.21.15.44:6801/2544087903","cluster_addr":"172.21.15.44:6803/2544087903","heartbeat_back_addr":"172.21.15.44:6807/2544087903","heartbeat_front_addr":"172.21.15.44:6805/2544087903","state":["exists","up"]},{"osd":3,"uuid":"bc514203-9af9-4f1f-b18e-765bdd5de6e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6825","nonce":1692928109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6827","nonce":1692928109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6831","nonce":1692928109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6829","nonce":1692928109}]},"public_addr":"172.21.15.44:6825/1692928109","cluster_addr":"172.21.15.44:6827/1692928109","heartbeat_back_addr":"172.21.15.44:6831/1692928109","heartbeat_front_addr":"172.21.15.44:6829/1692928109","state":["exists","up"]},{"osd":4,"uuid":"357c439a-6e08-4c19-ac18-b68e6c890497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6820","nonce":3802577427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6824","nonce":3802577427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6831","nonce":3802577427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6828","nonce":3802577427}]},"public_addr":"172.21.15.70:6820/3802577427","cluster_addr":"172.21.15.70:6824/3802577427","heartbeat_back_addr":"172.21.15.70:6831/3802577427","heartbeat_front_addr":"172.21.15.70:6828/3802577427","state":["exists","up"]},{"osd":5,"uuid":"952fb88f-d56e-4305-bfa8-d32674332dfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6801","nonce":664757345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6803","nonce":664757345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6807","nonce":664757345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6805","nonce":664757345}]},"public_addr":"172.21.15.70:6801/664757345","cluster_addr":"172.21.15.70:6803/664757345","heartbeat_back_addr":"172.21.15.70:6807/664757345","heartbeat_front_addr":"172.21.15.70:6805/664757345","state":["exists","up"]},{"osd":6,"uuid":"74399b3d-fac2-45ab-9172-2cd95b02450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6817","nonce":1623253482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6821","nonce":1623253482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6827","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6829","nonce":1623253482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6825","nonce":1623253482}]},"public_addr":"172.21.15.70:6817/1623253482","cluster_addr":"172.21.15.70:6821/1623253482","heartbeat_back_addr":"172.21.15.70:6829/1623253482","heartbeat_front_addr":"172.21.15.70:6825/1623253482","state":["exists","up"]},{"osd":7,"uuid":"8f35508d-6fcc-43ff-8ee7-0498a7ab3b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6809","nonce":1551180300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6811","nonce":1551180300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6815","nonce":1551180300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6813","nonce":1551180300}]},"public_addr":"172.21.15.70:6809/1551180300","cluster_addr":"172.21.15.70:6811/1551180300","heartbeat_back_addr":"172.21.15.70:6815/1551180300","heartbeat_front_addr":"172.21.15.70:6813/1551180300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.612502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.653756+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.521169+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.649955+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.640231+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.686638+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.642199+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.700360+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T06:31:29.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.679+0000 7f8a61515640 1 -- 172.21.15.44:0/2615088088 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f8a4806c920 msgr2=0x7f8a4806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:29.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.679+0000 7f8a61515640 1 --2- 172.21.15.44:0/2615088088 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f8a4806c920 0x7f8a4806ede0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c009b40 tx=0x7f8a4c009190 comp rx=0 tx=0).stop 2024-04-22T06:31:29.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.679+0000 7f8a61515640 1 -- 172.21.15.44:0/2615088088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a5c15c260 msgr2=0x7f8a5c1f4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:29.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.679+0000 7f8a61515640 1 --2- 172.21.15.44:0/2615088088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a5c15c260 0x7f8a5c1f4510 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8a5402c060 tx=0x7f8a540014a0 comp rx=0 tx=0).stop 2024-04-22T06:31:29.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.679+0000 7f8a61515640 1 -- 172.21.15.44:0/2615088088 shutdown_connections 2024-04-22T06:31:29.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.679+0000 7f8a61515640 1 --2- 172.21.15.44:0/2615088088 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f8a4806c920 0x7f8a4806ede0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:29.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.679+0000 7f8a61515640 1 --2- 172.21.15.44:0/2615088088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a5c15c260 0x7f8a5c1f4510 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:29.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.679+0000 7f8a61515640 1 --2- 172.21.15.44:0/2615088088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a5c0c7d00 0x7f8a5c1f3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:29.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.679+0000 7f8a61515640 1 --2- 172.21.15.44:0/2615088088 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8a440051c0 0x7f8a5c0d0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:29.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.680+0000 7f8a61515640 1 -- 172.21.15.44:0/2615088088 >> 172.21.15.44:0/2615088088 conn(0x7f8a5c075820 msgr2=0x7f8a5c1521f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:29.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.680+0000 7f8a61515640 1 -- 172.21.15.44:0/2615088088 shutdown_connections 2024-04-22T06:31:29.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.680+0000 7f8a61515640 1 -- 172.21.15.44:0/2615088088 wait complete. 2024-04-22T06:31:29.690 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T06:31:29.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.792+0000 7f0ca4411640 1 Processor -- start 2024-04-22T06:31:29.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.792+0000 7f0ca4411640 1 -- start start 2024-04-22T06:31:29.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.793+0000 7f0ca4411640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0c9c0cc4e0 0x7f0c9c15bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:29.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.793+0000 7f0ca4411640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c9c06c6e0 con 0x7f0c9c1552a0 2024-04-22T06:31:29.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.793+0000 7f0ca4411640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c9c06c9c0 con 0x7f0c9c155680 2024-04-22T06:31:29.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.793+0000 7f0ca4411640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f0c9c0cb990 con 0x7f0c9c0cc4e0 2024-04-22T06:31:29.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.793+0000 7f0ca21ad640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f0c9c1552a0 0x7f0c9c0cbdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:36686/0 (socket says 172.21.15.44:36686) 2024-04-22T06:31:29.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.793+0000 7f0ca21ad640 1 -- 172.21.15.44:0/3413069461 learned_addr learned my addr 172.21.15.44:0/3413069461 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:29.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.793+0000 7f0ca29ae640 1 --2- 172.21.15.44:0/3413069461 >> v2:172.21.15.70:3300/0 conn(0x7f0c9c0cc4e0 0x7f0c9c15bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:29.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.793+0000 7f0ca11ab640 1 -- 172.21.15.44:0/3413069461 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2104981850 0 0) 0x7f0c9c06c6e0 con 0x7f0c9c1552a0 2024-04-22T06:31:29.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.793+0000 7f0ca11ab640 1 -- 172.21.15.44:0/3413069461 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c88003520 con 0x7f0c9c1552a0 2024-04-22T06:31:29.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.793+0000 7f0ca11ab640 1 -- 172.21.15.44:0/3413069461 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3389722312 0 0) 0x7f0c88003520 con 0x7f0c9c1552a0 2024-04-22T06:31:29.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.793+0000 7f0ca11ab640 1 -- 172.21.15.44:0/3413069461 >> v2:172.21.15.70:3300/0 conn(0x7f0c9c0cc4e0 msgr2=0x7f0c9c15bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:29.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.793+0000 7f0ca11ab640 1 --2- 172.21.15.44:0/3413069461 >> v2:172.21.15.70:3300/0 conn(0x7f0c9c0cc4e0 0x7f0c9c15bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:29.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.793+0000 7f0ca11ab640 1 -- 172.21.15.44:0/3413069461 >> v1:172.21.15.70:6789/0 conn(0x7f0c9c155680 legacy=0x7f0c9c0ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:29.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.793+0000 7f0ca11ab640 1 -- 172.21.15.44:0/3413069461 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c9c06cca0 con 0x7f0c9c1552a0 2024-04-22T06:31:29.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.793+0000 7f0ca11ab640 1 -- 172.21.15.44:0/3413069461 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1237339257 0 0) 0x7f0c98001670 con 0x7f0c9c1552a0 2024-04-22T06:31:29.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.793+0000 7f0ca11ab640 1 -- 172.21.15.44:0/3413069461 >> v1:172.21.15.44:6789/0 conn(0x7f0c9c1552a0 legacy=0x7f0c9c0cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:29.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.793+0000 7f0ca11ab640 1 --2- 172.21.15.44:0/3413069461 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0c880039b0 0x7f0c88003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:29.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.793+0000 7f0ca11ab640 1 --2- 172.21.15.44:0/3413069461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c88004800 0x7f0c88004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:29.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.793+0000 7f0ca11ab640 1 --2- 172.21.15.44:0/3413069461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c880051c0 0x7f0c8800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:29.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.793+0000 7f0ca11ab640 1 -- 172.21.15.44:0/3413069461 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0c9c0cb990 con 0x7f0c88004800 2024-04-22T06:31:29.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.793+0000 7f0ca11ab640 1 -- 172.21.15.44:0/3413069461 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8800c070 con 0x7f0c880051c0 2024-04-22T06:31:29.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.793+0000 7f0ca11ab640 1 -- 172.21.15.44:0/3413069461 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f0c8800c270 con 0x7f0c880039b0 2024-04-22T06:31:29.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.793+0000 7f0ca19ac640 1 --2- 172.21.15.44:0/3413069461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c88004800 0x7f0c88004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:29.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.794+0000 7f0ca21ad640 1 --2- 172.21.15.44:0/3413069461 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0c880039b0 0x7f0c88003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:29.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.794+0000 7f0ca29ae640 1 --2- 172.21.15.44:0/3413069461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c880051c0 0x7f0c8800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:29.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.794+0000 7f0ca19ac640 1 -- 172.21.15.44:0/3413069461 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0c880039b0 msgr2=0x7f0c88003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:29.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.794+0000 7f0ca19ac640 1 --2- 172.21.15.44:0/3413069461 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0c880039b0 0x7f0c88003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:29.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.794+0000 7f0ca19ac640 1 -- 172.21.15.44:0/3413069461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c880051c0 msgr2=0x7f0c8800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:29.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.794+0000 7f0ca19ac640 1 --2- 172.21.15.44:0/3413069461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c880051c0 0x7f0c8800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:29.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.794+0000 7f0ca19ac640 1 -- 172.21.15.44:0/3413069461 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c88003520 con 0x7f0c88004800 2024-04-22T06:31:29.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.794+0000 7f0ca29ae640 1 --2- 172.21.15.44:0/3413069461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c880051c0 0x7f0c8800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:31:29.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.794+0000 7f0ca29ae640 1 -- 172.21.15.44:0/3413069461 reap_dead start 2024-04-22T06:31:29.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.794+0000 7f0ca19ac640 1 --2- 172.21.15.44:0/3413069461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c88004800 0x7f0c88004c80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0c90001750 tx=0x7f0c88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd9dc33a8a9b842 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:29.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.795+0000 7f0ca11ab640 1 -- 172.21.15.44:0/3413069461 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c90001370 con 0x7f0c88004800 2024-04-22T06:31:29.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.795+0000 7f0ca11ab640 1 -- 172.21.15.44:0/3413069461 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c900052d0 con 0x7f0c88004800 2024-04-22T06:31:29.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.795+0000 7f0ca11ab640 1 -- 172.21.15.44:0/3413069461 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c90003830 con 0x7f0c88004800 2024-04-22T06:31:29.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.795+0000 7f0ca4411640 1 -- 172.21.15.44:0/3413069461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c88004800 msgr2=0x7f0c88004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:29.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.795+0000 7f0ca4411640 1 --2- 172.21.15.44:0/3413069461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c88004800 0x7f0c88004c80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0c90001750 tx=0x7f0c88005830 comp rx=0 tx=0).stop 2024-04-22T06:31:29.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.795+0000 7f0ca4411640 1 -- 172.21.15.44:0/3413069461 shutdown_connections 2024-04-22T06:31:29.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.795+0000 7f0ca4411640 1 --2- 172.21.15.44:0/3413069461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c88004800 0x7f0c88004c80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:29.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.795+0000 7f0ca4411640 1 -- 172.21.15.44:0/3413069461 >> 172.21.15.44:0/3413069461 conn(0x7f0c9c075820 msgr2=0x7f0c9c073000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:29.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.795+0000 7f0ca4411640 1 -- 172.21.15.44:0/3413069461 shutdown_connections 2024-04-22T06:31:29.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.795+0000 7f0ca4411640 1 -- 172.21.15.44:0/3413069461 wait complete. 2024-04-22T06:31:29.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.796+0000 7f0ca4411640 1 Processor -- start 2024-04-22T06:31:29.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.796+0000 7f0ca4411640 1 -- start start 2024-04-22T06:31:29.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.797+0000 7f0ca4411640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0c88004800 0x7f0c9c1f1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:29.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.797+0000 7f0ca4411640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c9c157980 0x7f0c9c0ce800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:29.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.797+0000 7f0ca4411640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c9c0c5c00 0x7f0c9c0c65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:29.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.797+0000 7f0ca4411640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0c9c06d540 con 0x7f0c9c157980 2024-04-22T06:31:29.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.797+0000 7f0ca4411640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f0c9c06d260 con 0x7f0c9c0c5c00 2024-04-22T06:31:29.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.797+0000 7f0ca4411640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f0c9c0b6690 con 0x7f0c88004800 2024-04-22T06:31:29.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.797+0000 7f0ca19ac640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c9c157980 0x7f0c9c0ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:29.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.797+0000 7f0ca19ac640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c9c157980 0x7f0c9c0ce800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34246/0 (socket says 172.21.15.44:34246) 2024-04-22T06:31:29.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.797+0000 7f0ca19ac640 1 -- 172.21.15.44:0/1043929239 learned_addr learned my addr 172.21.15.44:0/1043929239 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:29.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.797+0000 7f0ca21ad640 1 --2- 172.21.15.44:0/1043929239 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0c88004800 0x7f0c9c1f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:29.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.797+0000 7f0ca29ae640 1 --2- 172.21.15.44:0/1043929239 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c9c0c5c00 0x7f0c9c0c65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:29.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.798+0000 7f0ca19ac640 1 -- 172.21.15.44:0/1043929239 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0c88004800 msgr2=0x7f0c9c1f1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:29.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.798+0000 7f0ca19ac640 1 --2- 172.21.15.44:0/1043929239 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0c88004800 0x7f0c9c1f1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:29.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.798+0000 7f0ca19ac640 1 -- 172.21.15.44:0/1043929239 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c9c0c5c00 msgr2=0x7f0c9c0c65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:29.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.798+0000 7f0ca19ac640 1 --2- 172.21.15.44:0/1043929239 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c9c0c5c00 0x7f0c9c0c65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:29.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.798+0000 7f0ca19ac640 1 -- 172.21.15.44:0/1043929239 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c9c158450 con 0x7f0c9c157980 2024-04-22T06:31:29.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.798+0000 7f0ca21ad640 1 --2- 172.21.15.44:0/1043929239 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0c88004800 0x7f0c9c1f1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:31:29.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.798+0000 7f0ca29ae640 1 --2- 172.21.15.44:0/1043929239 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c9c0c5c00 0x7f0c9c0c65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:31:29.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.798+0000 7f0ca19ac640 1 --2- 172.21.15.44:0/1043929239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c9c157980 0x7f0c9c0ce800 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0c90000c00 tx=0x7f0c90001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:29.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.798+0000 7f0c8effd640 1 -- 172.21.15.44:0/1043929239 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c90041020 con 0x7f0c9c157980 2024-04-22T06:31:29.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.798+0000 7f0c8effd640 1 -- 172.21.15.44:0/1043929239 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c90005480 con 0x7f0c9c157980 2024-04-22T06:31:29.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.798+0000 7f0ca4411640 1 -- 172.21.15.44:0/1043929239 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c9c158730 con 0x7f0c9c157980 2024-04-22T06:31:29.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.798+0000 7f0c8effd640 1 -- 172.21.15.44:0/1043929239 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c90007a30 con 0x7f0c9c157980 2024-04-22T06:31:29.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.798+0000 7f0ca4411640 1 -- 172.21.15.44:0/1043929239 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c9c0d05a0 con 0x7f0c9c157980 2024-04-22T06:31:29.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.799+0000 7f0ca4411640 1 -- 172.21.15.44:0/1043929239 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c9c068cc0 con 0x7f0c9c157980 2024-04-22T06:31:29.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.799+0000 7f0c8effd640 1 -- 172.21.15.44:0/1043929239 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0c90044020 con 0x7f0c9c157980 2024-04-22T06:31:29.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.800+0000 7f0c8effd640 1 --2- 172.21.15.44:0/1043929239 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f0c8006c8f0 0x7f0c8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:29.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.800+0000 7f0c8effd640 1 -- 172.21.15.44:0/1043929239 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0c900ad920 con 0x7f0c9c157980 2024-04-22T06:31:29.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.800+0000 7f0ca21ad640 1 --2- 172.21.15.44:0/1043929239 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f0c8006c8f0 0x7f0c8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:29.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.801+0000 7f0ca21ad640 1 --2- 172.21.15.44:0/1043929239 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f0c8006c8f0 0x7f0c8006edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0c98007d50 tx=0x7f0c98009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:29.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.804+0000 7f0c8effd640 1 -- 172.21.15.44:0/1043929239 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0c9004b030 con 0x7f0c9c157980 2024-04-22T06:31:29.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.989+0000 7f0ca4411640 1 -- 172.21.15.44:0/1043929239 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0c9c15a490 con 0x7f0c9c157980 2024-04-22T06:31:29.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.990+0000 7f0c8effd640 1 -- 172.21.15.44:0/1043929239 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18466 (secure 0 0 0) 0x7f0c9007b980 con 0x7f0c9c157980 2024-04-22T06:31:29.990 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:31:29.990 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"fsid":"9b0a34ca-403d-4513-8a66-d3749ba559aa","created":"2024-04-22T06:30:42.334581+0000","modified":"2024-04-22T06:31:11.406256+0000","last_up_change":"2024-04-22T06:30:52.942257+0000","last_in_change":"2024-04-22T06:30:44.467727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T06:30:54.560256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T06:31:04.612387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T06:31:05.312124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-22T06:31:09.025550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b4a6692-53c9-472d-b2f3-1dd29a6679ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6817","nonce":2220268493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6819","nonce":2220268493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6823","nonce":2220268493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6821","nonce":2220268493}]},"public_addr":"172.21.15.44:6817/2220268493","cluster_addr":"172.21.15.44:6819/2220268493","heartbeat_back_addr":"172.21.15.44:6823/2220268493","heartbeat_front_addr":"172.21.15.44:6821/2220268493","state":["exists","up"]},{"osd":1,"uuid":"70a6370a-7c0a-4d64-a0e4-b2e726b4f291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6809","nonce":1278678977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6811","nonce":1278678977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6815","nonce":1278678977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6813","nonce":1278678977}]},"public_addr":"172.21.15.44:6809/1278678977","cluster_addr":"172.21.15.44:6811/1278678977","heartbeat_back_addr":"172.21.15.44:6815/1278678977","heartbeat_front_addr":"172.21.15.44:6813/1278678977","state":["exists","up"]},{"osd":2,"uuid":"f2e51a62-ca91-470d-bd5a-5710c61deee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6801","nonce":2544087903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6803","nonce":2544087903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6807","nonce":2544087903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6805","nonce":2544087903}]},"public_addr":"172.21.15.44:6801/2544087903","cluster_addr":"172.21.15.44:6803/2544087903","heartbeat_back_addr":"172.21.15.44:6807/2544087903","heartbeat_front_addr":"172.21.15.44:6805/2544087903","state":["exists","up"]},{"osd":3,"uuid":"bc514203-9af9-4f1f-b18e-765bdd5de6e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6825","nonce":1692928109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6827","nonce":1692928109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6831","nonce":1692928109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6829","nonce":1692928109}]},"public_addr":"172.21.15.44:6825/1692928109","cluster_addr":"172.21.15.44:6827/1692928109","heartbeat_back_addr":"172.21.15.44:6831/1692928109","heartbeat_front_addr":"172.21.15.44:6829/1692928109","state":["exists","up"]},{"osd":4,"uuid":"357c439a-6e08-4c19-ac18-b68e6c890497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6820","nonce":3802577427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6824","nonce":3802577427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6831","nonce":3802577427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6828","nonce":3802577427}]},"public_addr":"172.21.15.70:6820/3802577427","cluster_addr":"172.21.15.70:6824/3802577427","heartbeat_back_addr":"172.21.15.70:6831/3802577427","heartbeat_front_addr":"172.21.15.70:6828/3802577427","state":["exists","up"]},{"osd":5,"uuid":"952fb88f-d56e-4305-bfa8-d32674332dfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6801","nonce":664757345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6803","nonce":664757345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6807","nonce":664757345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6805","nonce":664757345}]},"public_addr":"172.21.15.70:6801/664757345","cluster_addr":"172.21.15.70:6803/664757345","heartbeat_back_addr":"172.21.15.70:6807/664757345","heartbeat_front_addr":"172.21.15.70:6805/664757345","state":["exists","up"]},{"osd":6,"uuid":"74399b3d-fac2-45ab-9172-2cd95b02450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6817","nonce":1623253482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6821","nonce":1623253482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6827","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6829","nonce":1623253482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6825","nonce":1623253482}]},"public_addr":"172.21.15.70:6817/1623253482","cluster_addr":"172.21.15.70:6821/1623253482","heartbeat_back_addr":"172.21.15.70:6829/1623253482","heartbeat_front_addr":"172.21.15.70:6825/1623253482","state":["exists","up"]},{"osd":7,"uuid":"8f35508d-6fcc-43ff-8ee7-0498a7ab3b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6809","nonce":1551180300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6811","nonce":1551180300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6815","nonce":1551180300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6813","nonce":1551180300}]},"public_addr":"172.21.15.70:6809/1551180300","cluster_addr":"172.21.15.70:6811/1551180300","heartbeat_back_addr":"172.21.15.70:6815/1551180300","heartbeat_front_addr":"172.21.15.70:6813/1551180300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.612502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.653756+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.521169+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.649955+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.640231+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.686638+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.642199+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.700360+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T06:31:29.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.992+0000 7f0ca4411640 1 -- 172.21.15.44:0/1043929239 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f0c8006c8f0 msgr2=0x7f0c8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:29.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.992+0000 7f0ca4411640 1 --2- 172.21.15.44:0/1043929239 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f0c8006c8f0 0x7f0c8006edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0c98007d50 tx=0x7f0c98009040 comp rx=0 tx=0).stop 2024-04-22T06:31:29.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.992+0000 7f0ca4411640 1 -- 172.21.15.44:0/1043929239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c9c157980 msgr2=0x7f0c9c0ce800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:29.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.992+0000 7f0ca4411640 1 --2- 172.21.15.44:0/1043929239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c9c157980 0x7f0c9c0ce800 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0c90000c00 tx=0x7f0c90001120 comp rx=0 tx=0).stop 2024-04-22T06:31:29.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.992+0000 7f0ca4411640 1 -- 172.21.15.44:0/1043929239 shutdown_connections 2024-04-22T06:31:29.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.992+0000 7f0ca4411640 1 --2- 172.21.15.44:0/1043929239 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f0c8006c8f0 0x7f0c8006edb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:29.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.992+0000 7f0ca4411640 1 --2- 172.21.15.44:0/1043929239 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c9c0c5c00 0x7f0c9c0c65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:29.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.992+0000 7f0ca4411640 1 --2- 172.21.15.44:0/1043929239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c9c157980 0x7f0c9c0ce800 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:29.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.992+0000 7f0ca4411640 1 --2- 172.21.15.44:0/1043929239 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0c88004800 0x7f0c9c1f1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:29.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.992+0000 7f0ca4411640 1 -- 172.21.15.44:0/1043929239 >> 172.21.15.44:0/1043929239 conn(0x7f0c9c075820 msgr2=0x7f0c9c159090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:29.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.992+0000 7f0ca4411640 1 -- 172.21.15.44:0/1043929239 shutdown_connections 2024-04-22T06:31:29.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:29.992+0000 7f0ca4411640 1 -- 172.21.15.44:0/1043929239 wait complete. 2024-04-22T06:31:30.003 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-22T06:31:30.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.107+0000 7f3f252c8640 1 Processor -- start 2024-04-22T06:31:30.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.107+0000 7f3f252c8640 1 -- start start 2024-04-22T06:31:30.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.108+0000 7f3f252c8640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3f201551e0 0x7f3f2015bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:30.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.108+0000 7f3f252c8640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f2006b930 0x7f3f2006bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:30.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.108+0000 7f3f252c8640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f2006d4e0 con 0x7f3f20070ca0 2024-04-22T06:31:30.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.108+0000 7f3f252c8640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f3f20070810 con 0x7f3f201551e0 2024-04-22T06:31:30.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.108+0000 7f3f252c8640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3f20073380 con 0x7f3f2006b930 2024-04-22T06:31:30.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.109+0000 7f3f1effd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3f201551e0 0x7f3f2015bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:30.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.109+0000 7f3f1e7fc640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f2006b930 0x7f3f2006bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:30.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.109+0000 7f3f1e7fc640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f2006b930 0x7f3f2006bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.44:37476/0 (socket says 172.21.15.44:37476) 2024-04-22T06:31:30.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.109+0000 7f3f1effd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3f201551e0 0x7f3f2015bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:37058/0 (socket says 172.21.15.44:37058) 2024-04-22T06:31:30.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.109+0000 7f3f1e7fc640 1 -- 172.21.15.44:0/2436170753 learned_addr learned my addr 172.21.15.44:0/2436170753 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:30.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.109+0000 7f3f1dffb640 1 -- 172.21.15.44:0/2436170753 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 492528653 0 0) 0x7f3f2006d4e0 con 0x7f3f20070ca0 2024-04-22T06:31:30.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.109+0000 7f3f1dffb640 1 -- 172.21.15.44:0/2436170753 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f0c003520 con 0x7f3f20070ca0 2024-04-22T06:31:30.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.109+0000 7f3f1e7fc640 1 -- 172.21.15.44:0/2436170753 >> v2:172.21.15.70:3300/0 conn(0x7f3f201551e0 msgr2=0x7f3f2015bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:30.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.109+0000 7f3f1e7fc640 1 --2- 172.21.15.44:0/2436170753 >> v2:172.21.15.70:3300/0 conn(0x7f3f201551e0 0x7f3f2015bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:30.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.109+0000 7f3f1e7fc640 1 -- 172.21.15.44:0/2436170753 >> v1:172.21.15.70:6789/0 conn(0x7f3f20070ca0 legacy=0x7f3f200710f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:30.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.109+0000 7f3f1e7fc640 1 -- 172.21.15.44:0/2436170753 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f20073660 con 0x7f3f2006b930 2024-04-22T06:31:30.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.109+0000 7f3f1effd640 1 --2- 172.21.15.44:0/2436170753 >> v2:172.21.15.70:3300/0 conn(0x7f3f201551e0 0x7f3f2015bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:31:30.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.110+0000 7f3f1e7fc640 1 --2- 172.21.15.44:0/2436170753 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f2006b930 0x7f3f2006bd70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3f08009870 tx=0x7f3f0802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc24785c0c24acc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:30.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.110+0000 7f3f1dffb640 1 -- 172.21.15.44:0/2436170753 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f08034070 con 0x7f3f2006b930 2024-04-22T06:31:30.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.110+0000 7f3f1dffb640 1 -- 172.21.15.44:0/2436170753 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f08030840 con 0x7f3f2006b930 2024-04-22T06:31:30.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.110+0000 7f3f1dffb640 1 -- 172.21.15.44:0/2436170753 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f08030bf0 con 0x7f3f2006b930 2024-04-22T06:31:30.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.110+0000 7f3f252c8640 1 -- 172.21.15.44:0/2436170753 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f2006b930 msgr2=0x7f3f2006bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:30.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.110+0000 7f3f252c8640 1 --2- 172.21.15.44:0/2436170753 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f2006b930 0x7f3f2006bd70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3f08009870 tx=0x7f3f0802b980 comp rx=0 tx=0).stop 2024-04-22T06:31:30.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.111+0000 7f3f252c8640 1 -- 172.21.15.44:0/2436170753 shutdown_connections 2024-04-22T06:31:30.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.111+0000 7f3f252c8640 1 --2- 172.21.15.44:0/2436170753 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f2006b930 0x7f3f2006bd70 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:30.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.111+0000 7f3f252c8640 1 --2- 172.21.15.44:0/2436170753 >> v2:172.21.15.70:3300/0 conn(0x7f3f201551e0 0x7f3f2015bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:30.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.111+0000 7f3f252c8640 1 -- 172.21.15.44:0/2436170753 >> 172.21.15.44:0/2436170753 conn(0x7f3f200ce700 msgr2=0x7f3f200ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:30.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.111+0000 7f3f252c8640 1 -- 172.21.15.44:0/2436170753 shutdown_connections 2024-04-22T06:31:30.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.111+0000 7f3f252c8640 1 -- 172.21.15.44:0/2436170753 wait complete. 2024-04-22T06:31:30.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.112+0000 7f3f252c8640 1 Processor -- start 2024-04-22T06:31:30.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.112+0000 7f3f252c8640 1 -- start start 2024-04-22T06:31:30.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.113+0000 7f3f252c8640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f2006b930 0x7f3f20145650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:30.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.113+0000 7f3f252c8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f20070ca0 0x7f3f2014a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:30.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.113+0000 7f3f252c8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f201551e0 0x7f3f2014adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:30.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.113+0000 7f3f252c8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3f200cf420 con 0x7f3f20070ca0 2024-04-22T06:31:30.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.113+0000 7f3f252c8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f3f20073c20 con 0x7f3f201551e0 2024-04-22T06:31:30.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.113+0000 7f3f252c8640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3f201555c0 con 0x7f3f2006b930 2024-04-22T06:31:30.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.113+0000 7f3f1e7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f20070ca0 0x7f3f2014a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:30.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.113+0000 7f3f1e7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f20070ca0 0x7f3f2014a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34256/0 (socket says 172.21.15.44:34256) 2024-04-22T06:31:30.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.113+0000 7f3f1e7fc640 1 -- 172.21.15.44:0/2596414638 learned_addr learned my addr 172.21.15.44:0/2596414638 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:30.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.113+0000 7f3f1effd640 1 --2- 172.21.15.44:0/2596414638 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f2006b930 0x7f3f20145650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:30.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.113+0000 7f3f1f7fe640 1 --2- 172.21.15.44:0/2596414638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f201551e0 0x7f3f2014adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:30.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.113+0000 7f3f1e7fc640 1 -- 172.21.15.44:0/2596414638 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f2006b930 msgr2=0x7f3f20145650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:30.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.113+0000 7f3f1e7fc640 1 --2- 172.21.15.44:0/2596414638 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f2006b930 0x7f3f20145650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:30.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.113+0000 7f3f1e7fc640 1 -- 172.21.15.44:0/2596414638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f201551e0 msgr2=0x7f3f2014adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:30.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.113+0000 7f3f1e7fc640 1 --2- 172.21.15.44:0/2596414638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f201551e0 0x7f3f2014adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:30.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.113+0000 7f3f1e7fc640 1 -- 172.21.15.44:0/2596414638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f201f0e50 con 0x7f3f20070ca0 2024-04-22T06:31:30.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.113+0000 7f3f1e7fc640 1 --2- 172.21.15.44:0/2596414638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f20070ca0 0x7f3f2014a8b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3f080090b0 tx=0x7f3f080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:30.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.113+0000 7f3f177fe640 1 -- 172.21.15.44:0/2596414638 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f08047070 con 0x7f3f20070ca0 2024-04-22T06:31:30.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.113+0000 7f3f177fe640 1 -- 172.21.15.44:0/2596414638 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f08030e00 con 0x7f3f20070ca0 2024-04-22T06:31:30.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.113+0000 7f3f252c8640 1 -- 172.21.15.44:0/2596414638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f201f10d0 con 0x7f3f20070ca0 2024-04-22T06:31:30.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.113+0000 7f3f177fe640 1 -- 172.21.15.44:0/2596414638 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f08034040 con 0x7f3f20070ca0 2024-04-22T06:31:30.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.114+0000 7f3f252c8640 1 -- 172.21.15.44:0/2596414638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f201f1690 con 0x7f3f20070ca0 2024-04-22T06:31:30.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.115+0000 7f3f252c8640 1 -- 172.21.15.44:0/2596414638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f201f1d50 con 0x7f3f20070ca0 2024-04-22T06:31:30.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.115+0000 7f3f177fe640 1 -- 172.21.15.44:0/2596414638 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f0804a070 con 0x7f3f20070ca0 2024-04-22T06:31:30.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.115+0000 7f3f177fe640 1 --2- 172.21.15.44:0/2596414638 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f3f0006c8f0 0x7f3f0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:30.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.116+0000 7f3f177fe640 1 -- 172.21.15.44:0/2596414638 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3f080adac0 con 0x7f3f20070ca0 2024-04-22T06:31:30.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.116+0000 7f3f1effd640 1 --2- 172.21.15.44:0/2596414638 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f3f0006c8f0 0x7f3f0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:30.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.116+0000 7f3f1effd640 1 --2- 172.21.15.44:0/2596414638 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f3f0006c8f0 0x7f3f0006edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3f18009a30 tx=0x7f3f18008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:30.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.120+0000 7f3f177fe640 1 -- 172.21.15.44:0/2596414638 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3f201f1d50 con 0x7f3f20070ca0 2024-04-22T06:31:30.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:30.343+0000 7f3f252c8640 1 -- 172.21.15.44:0/2596414638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f3f20069310 con 0x7f3f20070ca0 2024-04-22T06:31:31.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.073+0000 7f3f177fe640 1 -- 172.21.15.44:0/2596414638 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7f3f201f0e50 con 0x7f3f20070ca0 2024-04-22T06:31:31.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.076+0000 7f3f252c8640 1 -- 172.21.15.44:0/2596414638 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f3f0006c8f0 msgr2=0x7f3f0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:31.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.076+0000 7f3f252c8640 1 --2- 172.21.15.44:0/2596414638 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f3f0006c8f0 0x7f3f0006edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3f18009a30 tx=0x7f3f18008040 comp rx=0 tx=0).stop 2024-04-22T06:31:31.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.076+0000 7f3f252c8640 1 -- 172.21.15.44:0/2596414638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f20070ca0 msgr2=0x7f3f2014a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:31.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.076+0000 7f3f252c8640 1 --2- 172.21.15.44:0/2596414638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f20070ca0 0x7f3f2014a8b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3f080090b0 tx=0x7f3f080046c0 comp rx=0 tx=0).stop 2024-04-22T06:31:31.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.076+0000 7f3f252c8640 1 -- 172.21.15.44:0/2596414638 shutdown_connections 2024-04-22T06:31:31.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.076+0000 7f3f252c8640 1 --2- 172.21.15.44:0/2596414638 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f3f0006c8f0 0x7f3f0006edb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:31.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.076+0000 7f3f252c8640 1 --2- 172.21.15.44:0/2596414638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f201551e0 0x7f3f2014adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:31.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.076+0000 7f3f252c8640 1 --2- 172.21.15.44:0/2596414638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f20070ca0 0x7f3f2014a8b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:31.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.076+0000 7f3f252c8640 1 --2- 172.21.15.44:0/2596414638 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f2006b930 0x7f3f20145650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:31.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.076+0000 7f3f252c8640 1 -- 172.21.15.44:0/2596414638 >> 172.21.15.44:0/2596414638 conn(0x7f3f200ce700 msgr2=0x7f3f20153f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:31.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.077+0000 7f3f252c8640 1 -- 172.21.15.44:0/2596414638 shutdown_connections 2024-04-22T06:31:31.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.077+0000 7f3f252c8640 1 -- 172.21.15.44:0/2596414638 wait complete. 2024-04-22T06:31:31.089 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300} 2024-04-22T06:31:31.090 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-04-22T06:31:31.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.200+0000 7fc122b52640 1 Processor -- start 2024-04-22T06:31:31.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.200+0000 7fc122b52640 1 -- start start 2024-04-22T06:31:31.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.201+0000 7fc122b52640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc11c154f00 0x7fc11c0cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:31.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.201+0000 7fc122b52640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc11c1552e0 0x7fc11c15c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:31.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.201+0000 7fc122b52640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc11c06d470 0x7fc11c0d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:31.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.201+0000 7fc122b52640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fc11c0c6740 con 0x7fc11c154f00 2024-04-22T06:31:31.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.201+0000 7fc122b52640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fc11c0c5b30 con 0x7fc11c1552e0 2024-04-22T06:31:31.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.201+0000 7fc122b52640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc11c0c5e10 con 0x7fc11c06d470 2024-04-22T06:31:31.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.201+0000 7fc1208ee640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc11c154f00 0x7fc11c0cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:31.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.201+0000 7fc1208ee640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc11c154f00 0x7fc11c0cb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34272/0 (socket says 172.21.15.44:34272) 2024-04-22T06:31:31.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.201+0000 7fc1208ee640 1 -- 172.21.15.44:0/1531514789 learned_addr learned my addr 172.21.15.44:0/1531514789 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:31.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.201+0000 7fc1210ef640 1 --2- 172.21.15.44:0/1531514789 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc11c06d470 0x7fc11c0d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:31.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.201+0000 7fc11bfff640 1 --2- 172.21.15.44:0/1531514789 >> v2:172.21.15.70:3300/0 conn(0x7fc11c1552e0 0x7fc11c15c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:31.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.201+0000 7fc1208ee640 1 -- 172.21.15.44:0/1531514789 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc11c06d470 msgr2=0x7fc11c0d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:31.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.201+0000 7fc1208ee640 1 --2- 172.21.15.44:0/1531514789 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc11c06d470 0x7fc11c0d0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:31.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.201+0000 7fc1208ee640 1 -- 172.21.15.44:0/1531514789 >> v2:172.21.15.70:3300/0 conn(0x7fc11c1552e0 msgr2=0x7fc11c15c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:31.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.201+0000 7fc1208ee640 1 --2- 172.21.15.44:0/1531514789 >> v2:172.21.15.70:3300/0 conn(0x7fc11c1552e0 0x7fc11c15c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:31.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.201+0000 7fc1208ee640 1 -- 172.21.15.44:0/1531514789 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc11c0c60f0 con 0x7fc11c154f00 2024-04-22T06:31:31.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.202+0000 7fc1208ee640 1 --2- 172.21.15.44:0/1531514789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc11c154f00 0x7fc11c0cb430 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc11c0ce560 tx=0x7fc11002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f077a085b9bc7fe2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:31.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.202+0000 7fc11b7fe640 1 -- 172.21.15.44:0/1531514789 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc110035020 con 0x7fc11c154f00 2024-04-22T06:31:31.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.202+0000 7fc11b7fe640 1 -- 172.21.15.44:0/1531514789 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc110031870 con 0x7fc11c154f00 2024-04-22T06:31:31.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.202+0000 7fc11b7fe640 1 -- 172.21.15.44:0/1531514789 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc110031c40 con 0x7fc11c154f00 2024-04-22T06:31:31.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.203+0000 7fc122b52640 1 -- 172.21.15.44:0/1531514789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc11c154f00 msgr2=0x7fc11c0cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:31.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.203+0000 7fc122b52640 1 --2- 172.21.15.44:0/1531514789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc11c154f00 0x7fc11c0cb430 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc11c0ce560 tx=0x7fc11002b910 comp rx=0 tx=0).stop 2024-04-22T06:31:31.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.203+0000 7fc122b52640 1 -- 172.21.15.44:0/1531514789 shutdown_connections 2024-04-22T06:31:31.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.203+0000 7fc122b52640 1 --2- 172.21.15.44:0/1531514789 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc11c06d470 0x7fc11c0d0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:31.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.203+0000 7fc122b52640 1 --2- 172.21.15.44:0/1531514789 >> v2:172.21.15.70:3300/0 conn(0x7fc11c1552e0 0x7fc11c15c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:31.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.203+0000 7fc122b52640 1 --2- 172.21.15.44:0/1531514789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc11c154f00 0x7fc11c0cb430 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:31.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.203+0000 7fc122b52640 1 -- 172.21.15.44:0/1531514789 >> 172.21.15.44:0/1531514789 conn(0x7fc11c0733c0 msgr2=0x7fc11c0737d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:31.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.203+0000 7fc122b52640 1 -- 172.21.15.44:0/1531514789 shutdown_connections 2024-04-22T06:31:31.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.203+0000 7fc122b52640 1 -- 172.21.15.44:0/1531514789 wait complete. 2024-04-22T06:31:31.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.204+0000 7fc122b52640 1 Processor -- start 2024-04-22T06:31:31.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.204+0000 7fc122b52640 1 -- start start 2024-04-22T06:31:31.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.205+0000 7fc122b52640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc11c06d470 0x7fc11c0b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:31.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.205+0000 7fc122b52640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc11c154f00 0x7fc11c0cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:31.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.205+0000 7fc122b52640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc11c1552e0 0x7fc11c159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:31.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.205+0000 7fc122b52640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc11c0c6d50 con 0x7fc11c1552e0 2024-04-22T06:31:31.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.205+0000 7fc122b52640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc11c0c6bd0 con 0x7fc11c06d470 2024-04-22T06:31:31.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.205+0000 7fc122b52640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc11c15cb20 con 0x7fc11c154f00 2024-04-22T06:31:31.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.205+0000 7fc1210ef640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc11c1552e0 0x7fc11c159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:31.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.205+0000 7fc1210ef640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc11c1552e0 0x7fc11c159690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34274/0 (socket says 172.21.15.44:34274) 2024-04-22T06:31:31.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.205+0000 7fc1210ef640 1 -- 172.21.15.44:0/2694498193 learned_addr learned my addr 172.21.15.44:0/2694498193 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:31.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.205+0000 7fc1208ee640 1 --2- 172.21.15.44:0/2694498193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc11c06d470 0x7fc11c0b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:31.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.205+0000 7fc11bfff640 1 --2- 172.21.15.44:0/2694498193 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc11c154f00 0x7fc11c0cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:31.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.205+0000 7fc1210ef640 1 -- 172.21.15.44:0/2694498193 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc11c154f00 msgr2=0x7fc11c0cdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:31.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.205+0000 7fc1210ef640 1 --2- 172.21.15.44:0/2694498193 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc11c154f00 0x7fc11c0cdb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:31.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.205+0000 7fc1210ef640 1 -- 172.21.15.44:0/2694498193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc11c06d470 msgr2=0x7fc11c0b5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:31.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.205+0000 7fc1210ef640 1 --2- 172.21.15.44:0/2694498193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc11c06d470 0x7fc11c0b5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:31.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.205+0000 7fc1210ef640 1 -- 172.21.15.44:0/2694498193 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc11c0cd6e0 con 0x7fc11c1552e0 2024-04-22T06:31:31.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.205+0000 7fc1208ee640 1 --2- 172.21.15.44:0/2694498193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc11c06d470 0x7fc11c0b5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:31:31.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.206+0000 7fc1210ef640 1 --2- 172.21.15.44:0/2694498193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc11c1552e0 0x7fc11c159690 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc11400ccf0 tx=0x7fc11400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:31.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.206+0000 7fc1197fa640 1 -- 172.21.15.44:0/2694498193 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc114003b90 con 0x7fc11c1552e0 2024-04-22T06:31:31.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.206+0000 7fc1197fa640 1 -- 172.21.15.44:0/2694498193 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc114003d30 con 0x7fc11c1552e0 2024-04-22T06:31:31.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.206+0000 7fc122b52640 1 -- 172.21.15.44:0/2694498193 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc11c0b49a0 con 0x7fc11c1552e0 2024-04-22T06:31:31.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.206+0000 7fc1197fa640 1 -- 172.21.15.44:0/2694498193 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc114004240 con 0x7fc11c1552e0 2024-04-22T06:31:31.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.206+0000 7fc122b52640 1 -- 172.21.15.44:0/2694498193 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc11c0b5680 con 0x7fc11c1552e0 2024-04-22T06:31:31.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.207+0000 7fc122b52640 1 -- 172.21.15.44:0/2694498193 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc11c068cc0 con 0x7fc11c1552e0 2024-04-22T06:31:31.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.207+0000 7fc1197fa640 1 -- 172.21.15.44:0/2694498193 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc1140043e0 con 0x7fc11c1552e0 2024-04-22T06:31:31.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.208+0000 7fc1197fa640 1 --2- 172.21.15.44:0/2694498193 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fc10006c8f0 0x7fc10006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:31.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.208+0000 7fc1197fa640 1 -- 172.21.15.44:0/2694498193 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc11408c890 con 0x7fc11c1552e0 2024-04-22T06:31:31.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.208+0000 7fc1208ee640 1 --2- 172.21.15.44:0/2694498193 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fc10006c8f0 0x7fc10006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:31.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.208+0000 7fc1208ee640 1 --2- 172.21.15.44:0/2694498193 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fc10006c8f0 0x7fc10006edb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc110036000 tx=0x7fc11003c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:31.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.212+0000 7fc1197fa640 1 -- 172.21.15.44:0/2694498193 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc11405a980 con 0x7fc11c1552e0 2024-04-22T06:31:31.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:31.436+0000 7fc122b52640 1 -- 172.21.15.44:0/2694498193 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7fc11c1517d0 con 0x7fc11c1552e0 2024-04-22T06:31:32.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.075+0000 7fc1197fa640 1 -- 172.21.15.44:0/2694498193 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v8) ==== 105+0+0 (secure 0 0 0) 0x7fc11405a280 con 0x7fc11c1552e0 2024-04-22T06:31:32.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.077+0000 7fc122b52640 1 -- 172.21.15.44:0/2694498193 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fc10006c8f0 msgr2=0x7fc10006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:32.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.077+0000 7fc122b52640 1 --2- 172.21.15.44:0/2694498193 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fc10006c8f0 0x7fc10006edb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc110036000 tx=0x7fc11003c040 comp rx=0 tx=0).stop 2024-04-22T06:31:32.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.077+0000 7fc122b52640 1 -- 172.21.15.44:0/2694498193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc11c1552e0 msgr2=0x7fc11c159690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:32.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.077+0000 7fc122b52640 1 --2- 172.21.15.44:0/2694498193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc11c1552e0 0x7fc11c159690 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc11400ccf0 tx=0x7fc11400a660 comp rx=0 tx=0).stop 2024-04-22T06:31:32.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.077+0000 7fc122b52640 1 -- 172.21.15.44:0/2694498193 shutdown_connections 2024-04-22T06:31:32.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.077+0000 7fc122b52640 1 --2- 172.21.15.44:0/2694498193 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fc10006c8f0 0x7fc10006edb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:32.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.077+0000 7fc122b52640 1 --2- 172.21.15.44:0/2694498193 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc11c154f00 0x7fc11c0cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:32.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.077+0000 7fc122b52640 1 --2- 172.21.15.44:0/2694498193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc11c1552e0 0x7fc11c159690 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:32.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.077+0000 7fc122b52640 1 --2- 172.21.15.44:0/2694498193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc11c06d470 0x7fc11c0b5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:32.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.077+0000 7fc122b52640 1 -- 172.21.15.44:0/2694498193 >> 172.21.15.44:0/2694498193 conn(0x7fc11c0733c0 msgr2=0x7fc11c1e4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:32.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.077+0000 7fc122b52640 1 -- 172.21.15.44:0/2694498193 shutdown_connections 2024-04-22T06:31:32.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.078+0000 7fc122b52640 1 -- 172.21.15.44:0/2694498193 wait complete. 2024-04-22T06:31:32.090 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-22T06:31:32.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.197+0000 7f3e0dfc7640 1 Processor -- start 2024-04-22T06:31:32.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.197+0000 7f3e0dfc7640 1 -- start start 2024-04-22T06:31:32.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.198+0000 7f3e0dfc7640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3e0815be00 0x7f3e080b4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:32.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.198+0000 7f3e0dfc7640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e080cc860 0x7f3e080b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:32.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.198+0000 7f3e0dfc7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e080cb8c0 con 0x7f3e080ca710 2024-04-22T06:31:32.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.198+0000 7f3e0dfc7640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f3e0815c360 con 0x7f3e0815be00 2024-04-22T06:31:32.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.198+0000 7f3e0dfc7640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3e080b3b00 con 0x7f3e080cc860 2024-04-22T06:31:32.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.198+0000 7f3e06ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3e0815be00 0x7f3e080b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:32.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.198+0000 7f3e06ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3e0815be00 0x7f3e080b4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53326/0 (socket says 172.21.15.44:53326) 2024-04-22T06:31:32.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.198+0000 7f3e06ffd640 1 -- 172.21.15.44:0/2849946989 learned_addr learned my addr 172.21.15.44:0/2849946989 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:32.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.198+0000 7f3e07fff640 1 --2- 172.21.15.44:0/2849946989 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e080cc860 0x7f3e080b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:32.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.198+0000 7f3e06ffd640 1 -- 172.21.15.44:0/2849946989 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e080cc860 msgr2=0x7f3e080b4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:32.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.198+0000 7f3e06ffd640 1 --2- 172.21.15.44:0/2849946989 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e080cc860 0x7f3e080b4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:32.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.198+0000 7f3e06ffd640 1 -- 172.21.15.44:0/2849946989 >> v1:172.21.15.70:6789/0 conn(0x7f3e080ca710 legacy=0x7f3e080d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:32.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.198+0000 7f3e06ffd640 1 -- 172.21.15.44:0/2849946989 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e080cbb60 con 0x7f3e0815be00 2024-04-22T06:31:32.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.199+0000 7f3e06ffd640 1 --2- 172.21.15.44:0/2849946989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e0815be00 0x7f3e080b4570 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3e080b3c80 tx=0x7f3dfc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b4b94b43103bb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:32.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.199+0000 7f3e067fc640 1 -- 172.21.15.44:0/2849946989 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc034070 con 0x7f3e0815be00 2024-04-22T06:31:32.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.199+0000 7f3e067fc640 1 -- 172.21.15.44:0/2849946989 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dfc030840 con 0x7f3e0815be00 2024-04-22T06:31:32.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.199+0000 7f3e067fc640 1 -- 172.21.15.44:0/2849946989 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc030c10 con 0x7f3e0815be00 2024-04-22T06:31:32.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.200+0000 7f3e0dfc7640 1 -- 172.21.15.44:0/2849946989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e0815be00 msgr2=0x7f3e080b4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:32.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.200+0000 7f3e0dfc7640 1 --2- 172.21.15.44:0/2849946989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e0815be00 0x7f3e080b4570 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3e080b3c80 tx=0x7f3dfc02b980 comp rx=0 tx=0).stop 2024-04-22T06:31:32.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.200+0000 7f3e0dfc7640 1 -- 172.21.15.44:0/2849946989 shutdown_connections 2024-04-22T06:31:32.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.200+0000 7f3e0dfc7640 1 --2- 172.21.15.44:0/2849946989 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e080cc860 0x7f3e080b4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:32.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.200+0000 7f3e0dfc7640 1 --2- 172.21.15.44:0/2849946989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e0815be00 0x7f3e080b4570 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:32.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.200+0000 7f3e0dfc7640 1 -- 172.21.15.44:0/2849946989 >> 172.21.15.44:0/2849946989 conn(0x7f3e0806d400 msgr2=0x7f3e08074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:32.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.200+0000 7f3e0dfc7640 1 -- 172.21.15.44:0/2849946989 shutdown_connections 2024-04-22T06:31:32.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.200+0000 7f3e0dfc7640 1 -- 172.21.15.44:0/2849946989 wait complete. 2024-04-22T06:31:32.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.201+0000 7f3e0dfc7640 1 Processor -- start 2024-04-22T06:31:32.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.201+0000 7f3e0dfc7640 1 -- start start 2024-04-22T06:31:32.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.202+0000 7f3e0dfc7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e080ca710 0x7f3e081f4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:32.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.202+0000 7f3e0dfc7640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e080cc860 0x7f3e081f0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:32.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.202+0000 7f3e0dfc7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e0815be00 0x7f3e080d0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:32.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.202+0000 7f3e0dfc7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3e080b3670 con 0x7f3e080ca710 2024-04-22T06:31:32.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.202+0000 7f3e0dfc7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f3e080b4030 con 0x7f3e0815be00 2024-04-22T06:31:32.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.202+0000 7f3e0dfc7640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3e080d10b0 con 0x7f3e080cc860 2024-04-22T06:31:32.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.202+0000 7f3e077fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e080ca710 0x7f3e081f4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:32.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.202+0000 7f3e077fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e080ca710 0x7f3e081f4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53334/0 (socket says 172.21.15.44:53334) 2024-04-22T06:31:32.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.202+0000 7f3e077fe640 1 -- 172.21.15.44:0/1230234459 learned_addr learned my addr 172.21.15.44:0/1230234459 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:32.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.202+0000 7f3e07fff640 1 --2- 172.21.15.44:0/1230234459 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e0815be00 0x7f3e080d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:32.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.202+0000 7f3e06ffd640 1 --2- 172.21.15.44:0/1230234459 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e080cc860 0x7f3e081f0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:32.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.202+0000 7f3e077fe640 1 -- 172.21.15.44:0/1230234459 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e080cc860 msgr2=0x7f3e081f0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:32.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.202+0000 7f3e077fe640 1 --2- 172.21.15.44:0/1230234459 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e080cc860 0x7f3e081f0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:32.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.202+0000 7f3e077fe640 1 -- 172.21.15.44:0/1230234459 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e0815be00 msgr2=0x7f3e080d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:32.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.202+0000 7f3e077fe640 1 --2- 172.21.15.44:0/1230234459 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e0815be00 0x7f3e080d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:32.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.202+0000 7f3e077fe640 1 -- 172.21.15.44:0/1230234459 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e08151950 con 0x7f3e080ca710 2024-04-22T06:31:32.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.202+0000 7f3e077fe640 1 --2- 172.21.15.44:0/1230234459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e080ca710 0x7f3e081f4b20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3df800c430 tx=0x7f3df800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:32.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.202+0000 7f3deffff640 1 -- 172.21.15.44:0/1230234459 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df80052d0 con 0x7f3e080ca710 2024-04-22T06:31:32.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.202+0000 7f3deffff640 1 -- 172.21.15.44:0/1230234459 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3df8005490 con 0x7f3e080ca710 2024-04-22T06:31:32.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.203+0000 7f3e0dfc7640 1 -- 172.21.15.44:0/1230234459 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e08158390 con 0x7f3e080ca710 2024-04-22T06:31:32.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.203+0000 7f3deffff640 1 -- 172.21.15.44:0/1230234459 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df8007260 con 0x7f3e080ca710 2024-04-22T06:31:32.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.203+0000 7f3e0dfc7640 1 -- 172.21.15.44:0/1230234459 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e08158630 con 0x7f3e080ca710 2024-04-22T06:31:32.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.204+0000 7f3e0dfc7640 1 -- 172.21.15.44:0/1230234459 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e08068cc0 con 0x7f3e080ca710 2024-04-22T06:31:32.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.204+0000 7f3deffff640 1 -- 172.21.15.44:0/1230234459 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3df8007400 con 0x7f3e080ca710 2024-04-22T06:31:32.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.204+0000 7f3deffff640 1 --2- 172.21.15.44:0/1230234459 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f3df406c8f0 0x7f3df406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:32.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.205+0000 7f3deffff640 1 -- 172.21.15.44:0/1230234459 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3df808b480 con 0x7f3e080ca710 2024-04-22T06:31:32.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.205+0000 7f3e06ffd640 1 --2- 172.21.15.44:0/1230234459 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f3df406c8f0 0x7f3df406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:32.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.205+0000 7f3e06ffd640 1 --2- 172.21.15.44:0/1230234459 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f3df406c8f0 0x7f3df406edb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc036000 tx=0x7f3dfc03b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:32.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.209+0000 7f3deffff640 1 -- 172.21.15.44:0/1230234459 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3e08151950 con 0x7f3e080ca710 2024-04-22T06:31:32.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:32.433+0000 7f3e0dfc7640 1 -- 172.21.15.44:0/1230234459 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f3e080cc260 con 0x7f3e080ca710 2024-04-22T06:31:33.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.082+0000 7f3deffff640 1 -- 172.21.15.44:0/1230234459 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f3df80594e0 con 0x7f3e080ca710 2024-04-22T06:31:33.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.085+0000 7f3e0dfc7640 1 -- 172.21.15.44:0/1230234459 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f3df406c8f0 msgr2=0x7f3df406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:33.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.085+0000 7f3e0dfc7640 1 --2- 172.21.15.44:0/1230234459 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f3df406c8f0 0x7f3df406edb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc036000 tx=0x7f3dfc03b040 comp rx=0 tx=0).stop 2024-04-22T06:31:33.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.085+0000 7f3e0dfc7640 1 -- 172.21.15.44:0/1230234459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e080ca710 msgr2=0x7f3e081f4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:33.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.085+0000 7f3e0dfc7640 1 --2- 172.21.15.44:0/1230234459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e080ca710 0x7f3e081f4b20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3df800c430 tx=0x7f3df800c8f0 comp rx=0 tx=0).stop 2024-04-22T06:31:33.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.085+0000 7f3e0dfc7640 1 -- 172.21.15.44:0/1230234459 shutdown_connections 2024-04-22T06:31:33.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.085+0000 7f3e0dfc7640 1 --2- 172.21.15.44:0/1230234459 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f3df406c8f0 0x7f3df406edb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:33.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.085+0000 7f3e0dfc7640 1 --2- 172.21.15.44:0/1230234459 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e080cc860 0x7f3e081f0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:33.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.085+0000 7f3e0dfc7640 1 --2- 172.21.15.44:0/1230234459 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e0815be00 0x7f3e080d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:33.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.085+0000 7f3e0dfc7640 1 --2- 172.21.15.44:0/1230234459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e080ca710 0x7f3e081f4b20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:33.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.085+0000 7f3e0dfc7640 1 -- 172.21.15.44:0/1230234459 >> 172.21.15.44:0/1230234459 conn(0x7f3e0806d400 msgr2=0x7f3e080b4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:33.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.086+0000 7f3e0dfc7640 1 -- 172.21.15.44:0/1230234459 shutdown_connections 2024-04-22T06:31:33.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.086+0000 7f3e0dfc7640 1 -- 172.21.15.44:0/1230234459 wait complete. 2024-04-22T06:31:33.099 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-22T06:31:33.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.210+0000 7f5609b3d640 1 Processor -- start 2024-04-22T06:31:33.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.210+0000 7f5609b3d640 1 -- start start 2024-04-22T06:31:33.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.211+0000 7f5609b3d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56040c5c00 0x7f56040c6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:33.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.211+0000 7f5609b3d640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56040ccde0 con 0x7f5604155460 2024-04-22T06:31:33.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.211+0000 7f5609b3d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56040c6570 con 0x7f560415f050 2024-04-22T06:31:33.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.211+0000 7f5609b3d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f5604155840 con 0x7f56040c5c00 2024-04-22T06:31:33.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.211+0000 7f56037fe640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f5604155460 0x7f56040cc6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:49398/0 (socket says 172.21.15.44:49398) 2024-04-22T06:31:33.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.211+0000 7f56037fe640 1 -- 172.21.15.44:0/2945074591 learned_addr learned my addr 172.21.15.44:0/2945074591 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:33.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.211+0000 7f5603fff640 1 --2- 172.21.15.44:0/2945074591 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56040c5c00 0x7f56040c6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:33.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.211+0000 7f56027fc640 1 -- 172.21.15.44:0/2945074591 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3519159639 0 0) 0x7f56040ccde0 con 0x7f5604155460 2024-04-22T06:31:33.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.211+0000 7f56027fc640 1 -- 172.21.15.44:0/2945074591 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55ec003520 con 0x7f5604155460 2024-04-22T06:31:33.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.211+0000 7f56027fc640 1 -- 172.21.15.44:0/2945074591 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 872338794 0 0) 0x7f56040c6570 con 0x7f560415f050 2024-04-22T06:31:33.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.211+0000 7f56027fc640 1 -- 172.21.15.44:0/2945074591 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f56040ccde0 con 0x7f560415f050 2024-04-22T06:31:33.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.211+0000 7f56027fc640 1 -- 172.21.15.44:0/2945074591 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 214186169 0 0) 0x7f55ec003520 con 0x7f5604155460 2024-04-22T06:31:33.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.211+0000 7f56027fc640 1 -- 172.21.15.44:0/2945074591 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56040c5c00 msgr2=0x7f56040c6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:33.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.211+0000 7f56027fc640 1 --2- 172.21.15.44:0/2945074591 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56040c5c00 0x7f56040c6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:33.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.211+0000 7f56027fc640 1 -- 172.21.15.44:0/2945074591 >> v1:172.21.15.70:6789/0 conn(0x7f560415f050 legacy=0x7f5604073230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:33.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.211+0000 7f56027fc640 1 -- 172.21.15.44:0/2945074591 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56040cd0a0 con 0x7f5604155460 2024-04-22T06:31:33.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.211+0000 7f56027fc640 1 -- 172.21.15.44:0/2945074591 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1237339257 0 0) 0x7f55f4001670 con 0x7f5604155460 2024-04-22T06:31:33.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.211+0000 7f56027fc640 1 -- 172.21.15.44:0/2945074591 >> v1:172.21.15.44:6789/0 conn(0x7f5604155460 legacy=0x7f56040cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:33.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.211+0000 7f56027fc640 1 --2- 172.21.15.44:0/2945074591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55ec003b70 0x7f55ec003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:33.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.211+0000 7f56027fc640 1 --2- 172.21.15.44:0/2945074591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55ec004800 0x7f55ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:33.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.212+0000 7f56027fc640 1 --2- 172.21.15.44:0/2945074591 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56040c5c00 0x7f55ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:33.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.212+0000 7f56027fc640 1 -- 172.21.15.44:0/2945074591 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5604155840 con 0x7f55ec004800 2024-04-22T06:31:33.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.212+0000 7f56027fc640 1 -- 172.21.15.44:0/2945074591 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f55ec00ba70 con 0x7f55ec003b70 2024-04-22T06:31:33.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.212+0000 7f56027fc640 1 -- 172.21.15.44:0/2945074591 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f55ec00bc70 con 0x7f56040c5c00 2024-04-22T06:31:33.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.212+0000 7f5602ffd640 1 --2- 172.21.15.44:0/2945074591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55ec004800 0x7f55ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:33.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.212+0000 7f5603fff640 1 --2- 172.21.15.44:0/2945074591 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56040c5c00 0x7f55ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:33.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.212+0000 7f56037fe640 1 --2- 172.21.15.44:0/2945074591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55ec003b70 0x7f55ec003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:33.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.212+0000 7f5602ffd640 1 -- 172.21.15.44:0/2945074591 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56040c5c00 msgr2=0x7f55ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:33.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.212+0000 7f5602ffd640 1 --2- 172.21.15.44:0/2945074591 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56040c5c00 0x7f55ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:33.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.212+0000 7f5602ffd640 1 -- 172.21.15.44:0/2945074591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55ec003b70 msgr2=0x7f55ec003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:33.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.212+0000 7f5602ffd640 1 --2- 172.21.15.44:0/2945074591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55ec003b70 0x7f55ec003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:33.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.212+0000 7f5602ffd640 1 -- 172.21.15.44:0/2945074591 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55ec003520 con 0x7f55ec004800 2024-04-22T06:31:33.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.213+0000 7f5602ffd640 1 --2- 172.21.15.44:0/2945074591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55ec004800 0x7f55ec004c80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f55f80018d0 tx=0x7f55ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad4fea8bfe222f6f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:33.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.213+0000 7f56027fc640 1 -- 172.21.15.44:0/2945074591 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f8037070 con 0x7f55ec004800 2024-04-22T06:31:33.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.213+0000 7f56027fc640 1 -- 172.21.15.44:0/2945074591 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55f80052d0 con 0x7f55ec004800 2024-04-22T06:31:33.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.213+0000 7f56027fc640 1 -- 172.21.15.44:0/2945074591 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f80014f0 con 0x7f55ec004800 2024-04-22T06:31:33.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.213+0000 7f5609b3d640 1 -- 172.21.15.44:0/2945074591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55ec004800 msgr2=0x7f55ec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:33.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.213+0000 7f5609b3d640 1 --2- 172.21.15.44:0/2945074591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55ec004800 0x7f55ec004c80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f55f80018d0 tx=0x7f55ec005830 comp rx=0 tx=0).stop 2024-04-22T06:31:33.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.213+0000 7f5603fff640 1 -- 172.21.15.44:0/2945074591 reap_dead start 2024-04-22T06:31:33.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.213+0000 7f5609b3d640 1 -- 172.21.15.44:0/2945074591 shutdown_connections 2024-04-22T06:31:33.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.213+0000 7f5609b3d640 1 -- 172.21.15.44:0/2945074591 >> 172.21.15.44:0/2945074591 conn(0x7f560406cca0 msgr2=0x7f560406d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:33.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.213+0000 7f5609b3d640 1 -- 172.21.15.44:0/2945074591 shutdown_connections 2024-04-22T06:31:33.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.213+0000 7f5609b3d640 1 -- 172.21.15.44:0/2945074591 wait complete. 2024-04-22T06:31:33.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.215+0000 7f5609b3d640 1 Processor -- start 2024-04-22T06:31:33.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.215+0000 7f5609b3d640 1 -- start start 2024-04-22T06:31:33.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.215+0000 7f5609b3d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55ec004800 0x7f56040c8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:33.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.215+0000 7f5609b3d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f560415c210 0x7f56040ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:33.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.215+0000 7f5609b3d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5604147640 0x7f560415e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:33.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.216+0000 7f5609b3d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f56040d0bf0 con 0x7f55ec004800 2024-04-22T06:31:33.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.216+0000 7f5609b3d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f56040d1520 con 0x7f5604147640 2024-04-22T06:31:33.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.216+0000 7f5609b3d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f56040d0ed0 con 0x7f560415c210 2024-04-22T06:31:33.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.216+0000 7f56037fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55ec004800 0x7f56040c8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:33.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.216+0000 7f56037fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55ec004800 0x7f56040c8c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53344/0 (socket says 172.21.15.44:53344) 2024-04-22T06:31:33.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.216+0000 7f56037fe640 1 -- 172.21.15.44:0/1619867727 learned_addr learned my addr 172.21.15.44:0/1619867727 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:33.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.216+0000 7f5602ffd640 1 --2- 172.21.15.44:0/1619867727 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f560415c210 0x7f56040ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:33.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.216+0000 7f5603fff640 1 --2- 172.21.15.44:0/1619867727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5604147640 0x7f560415e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:33.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.216+0000 7f56037fe640 1 -- 172.21.15.44:0/1619867727 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f560415c210 msgr2=0x7f56040ca6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:33.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.216+0000 7f56037fe640 1 --2- 172.21.15.44:0/1619867727 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f560415c210 0x7f56040ca6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:33.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.216+0000 7f56037fe640 1 -- 172.21.15.44:0/1619867727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5604147640 msgr2=0x7f560415e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:33.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.216+0000 7f56037fe640 1 --2- 172.21.15.44:0/1619867727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5604147640 0x7f560415e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:33.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.216+0000 7f56037fe640 1 -- 172.21.15.44:0/1619867727 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56040c92d0 con 0x7f55ec004800 2024-04-22T06:31:33.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.216+0000 7f5603fff640 1 --2- 172.21.15.44:0/1619867727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5604147640 0x7f560415e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:31:33.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.216+0000 7f56037fe640 1 --2- 172.21.15.44:0/1619867727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55ec004800 0x7f56040c8c90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f55f4004f50 tx=0x7f55f40074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:33.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.216+0000 7f55ebfff640 1 -- 172.21.15.44:0/1619867727 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f401a070 con 0x7f55ec004800 2024-04-22T06:31:33.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.216+0000 7f55ebfff640 1 -- 172.21.15.44:0/1619867727 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55f4007740 con 0x7f55ec004800 2024-04-22T06:31:33.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.216+0000 7f5609b3d640 1 -- 172.21.15.44:0/1619867727 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56040c95b0 con 0x7f55ec004800 2024-04-22T06:31:33.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.216+0000 7f55ebfff640 1 -- 172.21.15.44:0/1619867727 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f400fb00 con 0x7f55ec004800 2024-04-22T06:31:33.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.216+0000 7f5609b3d640 1 -- 172.21.15.44:0/1619867727 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56040c9ed0 con 0x7f55ec004800 2024-04-22T06:31:33.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.217+0000 7f5609b3d640 1 -- 172.21.15.44:0/1619867727 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5604068cc0 con 0x7f55ec004800 2024-04-22T06:31:33.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.218+0000 7f55ebfff640 1 -- 172.21.15.44:0/1619867727 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f55f401d070 con 0x7f55ec004800 2024-04-22T06:31:33.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.218+0000 7f55ebfff640 1 --2- 172.21.15.44:0/1619867727 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f55f006c8f0 0x7f55f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:33.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.218+0000 7f55ebfff640 1 -- 172.21.15.44:0/1619867727 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f55f408b770 con 0x7f55ec004800 2024-04-22T06:31:33.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.218+0000 7f5602ffd640 1 --2- 172.21.15.44:0/1619867727 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f55f006c8f0 0x7f55f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:33.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.219+0000 7f5602ffd640 1 --2- 172.21.15.44:0/1619867727 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f55f006c8f0 0x7f55f006edb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f55f8034000 tx=0x7f55f8031000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:33.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.222+0000 7f55ebfff640 1 -- 172.21.15.44:0/1619867727 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f55f4059860 con 0x7f55ec004800 2024-04-22T06:31:33.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:33.447+0000 7f5609b3d640 1 -- 172.21.15.44:0/1619867727 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f56040c8a70 con 0x7f55ec004800 2024-04-22T06:31:34.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.084+0000 7f55ebfff640 1 -- 172.21.15.44:0/1619867727 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v11) ==== 115+0+0 (secure 0 0 0) 0x7f55f4059160 con 0x7f55ec004800 2024-04-22T06:31:34.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.087+0000 7f5609b3d640 1 -- 172.21.15.44:0/1619867727 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f55f006c8f0 msgr2=0x7f55f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:34.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.087+0000 7f5609b3d640 1 --2- 172.21.15.44:0/1619867727 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f55f006c8f0 0x7f55f006edb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f55f8034000 tx=0x7f55f8031000 comp rx=0 tx=0).stop 2024-04-22T06:31:34.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.087+0000 7f5609b3d640 1 -- 172.21.15.44:0/1619867727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55ec004800 msgr2=0x7f56040c8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:34.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.087+0000 7f5609b3d640 1 --2- 172.21.15.44:0/1619867727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55ec004800 0x7f56040c8c90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f55f4004f50 tx=0x7f55f40074f0 comp rx=0 tx=0).stop 2024-04-22T06:31:34.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.087+0000 7f5609b3d640 1 -- 172.21.15.44:0/1619867727 shutdown_connections 2024-04-22T06:31:34.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.087+0000 7f5609b3d640 1 --2- 172.21.15.44:0/1619867727 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f55f006c8f0 0x7f55f006edb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:34.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.087+0000 7f5609b3d640 1 --2- 172.21.15.44:0/1619867727 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f560415c210 0x7f56040ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:34.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.087+0000 7f5609b3d640 1 --2- 172.21.15.44:0/1619867727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5604147640 0x7f560415e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:34.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.087+0000 7f5609b3d640 1 --2- 172.21.15.44:0/1619867727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55ec004800 0x7f56040c8c90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:34.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.087+0000 7f5609b3d640 1 -- 172.21.15.44:0/1619867727 >> 172.21.15.44:0/1619867727 conn(0x7f560406cca0 msgr2=0x7f5604154340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:34.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.087+0000 7f5609b3d640 1 -- 172.21.15.44:0/1619867727 shutdown_connections 2024-04-22T06:31:34.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.087+0000 7f5609b3d640 1 -- 172.21.15.44:0/1619867727 wait complete. 2024-04-22T06:31:34.101 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-22T06:31:34.101 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T06:31:34.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.208+0000 7f2c7b511640 1 Processor -- start 2024-04-22T06:31:34.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.208+0000 7f2c7b511640 1 -- start start 2024-04-22T06:31:34.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.209+0000 7f2c7b511640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2c74155680 0x7f2c7415bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:34.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.209+0000 7f2c7b511640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2c740cc4e0 0x7f2c740ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:34.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.209+0000 7f2c7b511640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c7406c6e0 con 0x7f2c741552a0 2024-04-22T06:31:34.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.209+0000 7f2c7b511640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f2c740cb990 con 0x7f2c740cc4e0 2024-04-22T06:31:34.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.209+0000 7f2c7b511640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f2c7406c9c0 con 0x7f2c74155680 2024-04-22T06:31:34.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.209+0000 7f2c79aae640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2c740cc4e0 0x7f2c740ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:34.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.209+0000 7f2c792ad640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f2c741552a0 0x7f2c740cbdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:49404/0 (socket says 172.21.15.44:49404) 2024-04-22T06:31:34.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.209+0000 7f2c792ad640 1 -- 172.21.15.44:0/2340668799 learned_addr learned my addr 172.21.15.44:0/2340668799 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:34.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.209+0000 7f2c78aac640 1 --2- 172.21.15.44:0/2340668799 >> v2:172.21.15.70:3300/0 conn(0x7f2c74155680 0x7f2c7415bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:34.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.209+0000 7f2c79aae640 1 -- 172.21.15.44:0/2340668799 >> v2:172.21.15.70:3300/0 conn(0x7f2c74155680 msgr2=0x7f2c7415bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:34.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.209+0000 7f2c79aae640 1 --2- 172.21.15.44:0/2340668799 >> v2:172.21.15.70:3300/0 conn(0x7f2c74155680 0x7f2c7415bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:34.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.209+0000 7f2c79aae640 1 -- 172.21.15.44:0/2340668799 >> v1:172.21.15.44:6789/0 conn(0x7f2c741552a0 legacy=0x7f2c740cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:34.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.209+0000 7f2c79aae640 1 -- 172.21.15.44:0/2340668799 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c7406cca0 con 0x7f2c740cc4e0 2024-04-22T06:31:34.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.209+0000 7f2c6bfff640 1 -- 172.21.15.44:0/2340668799 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 99984649 0 0) 0x7f2c7406c6e0 con 0x7f2c741552a0 2024-04-22T06:31:34.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.209+0000 7f2c78aac640 1 --2- 172.21.15.44:0/2340668799 >> v2:172.21.15.70:3300/0 conn(0x7f2c74155680 0x7f2c7415bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:31:34.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.210+0000 7f2c79aae640 1 --2- 172.21.15.44:0/2340668799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c740cc4e0 0x7f2c740ccfc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2c740ce480 tx=0x7f2c6c02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b027af533ab091 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:34.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.210+0000 7f2c6bfff640 1 -- 172.21.15.44:0/2340668799 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c039070 con 0x7f2c740cc4e0 2024-04-22T06:31:34.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.210+0000 7f2c6bfff640 1 -- 172.21.15.44:0/2340668799 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c6c02cd00 con 0x7f2c740cc4e0 2024-04-22T06:31:34.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.210+0000 7f2c6bfff640 1 -- 172.21.15.44:0/2340668799 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c034660 con 0x7f2c740cc4e0 2024-04-22T06:31:34.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.211+0000 7f2c7b511640 1 -- 172.21.15.44:0/2340668799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c740cc4e0 msgr2=0x7f2c740ccfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:34.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.211+0000 7f2c7b511640 1 --2- 172.21.15.44:0/2340668799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c740cc4e0 0x7f2c740ccfc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2c740ce480 tx=0x7f2c6c02bf50 comp rx=0 tx=0).stop 2024-04-22T06:31:34.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.211+0000 7f2c7b511640 1 -- 172.21.15.44:0/2340668799 shutdown_connections 2024-04-22T06:31:34.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.211+0000 7f2c7b511640 1 --2- 172.21.15.44:0/2340668799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c740cc4e0 0x7f2c740ccfc0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:34.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.211+0000 7f2c7b511640 1 --2- 172.21.15.44:0/2340668799 >> v2:172.21.15.70:3300/0 conn(0x7f2c74155680 0x7f2c7415bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:34.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.211+0000 7f2c7b511640 1 -- 172.21.15.44:0/2340668799 >> 172.21.15.44:0/2340668799 conn(0x7f2c74075820 msgr2=0x7f2c74073000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:34.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.211+0000 7f2c7b511640 1 -- 172.21.15.44:0/2340668799 shutdown_connections 2024-04-22T06:31:34.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.211+0000 7f2c7b511640 1 -- 172.21.15.44:0/2340668799 wait complete. 2024-04-22T06:31:34.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.212+0000 7f2c7b511640 1 Processor -- start 2024-04-22T06:31:34.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.212+0000 7f2c7b511640 1 -- start start 2024-04-22T06:31:34.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.213+0000 7f2c7b511640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c740cc4e0 0x7f2c740c5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:34.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.213+0000 7f2c7b511640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c741552a0 0x7f2c74153e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:34.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.213+0000 7f2c7b511640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c740c7030 0x7f2c74150a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:34.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.213+0000 7f2c7b511640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7406d540 con 0x7f2c741552a0 2024-04-22T06:31:34.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.213+0000 7f2c7b511640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7406d260 con 0x7f2c740c7030 2024-04-22T06:31:34.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.213+0000 7f2c7b511640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2c740b6690 con 0x7f2c740cc4e0 2024-04-22T06:31:34.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.213+0000 7f2c78aac640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c741552a0 0x7f2c74153e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:34.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.213+0000 7f2c78aac640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c741552a0 0x7f2c74153e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53366/0 (socket says 172.21.15.44:53366) 2024-04-22T06:31:34.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.213+0000 7f2c78aac640 1 -- 172.21.15.44:0/1045018231 learned_addr learned my addr 172.21.15.44:0/1045018231 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:34.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.213+0000 7f2c792ad640 1 --2- 172.21.15.44:0/1045018231 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c740cc4e0 0x7f2c740c5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:34.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.213+0000 7f2c79aae640 1 --2- 172.21.15.44:0/1045018231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c740c7030 0x7f2c74150a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:34.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.213+0000 7f2c78aac640 1 -- 172.21.15.44:0/1045018231 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c740cc4e0 msgr2=0x7f2c740c5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:34.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.213+0000 7f2c78aac640 1 --2- 172.21.15.44:0/1045018231 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c740cc4e0 0x7f2c740c5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:34.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.213+0000 7f2c78aac640 1 -- 172.21.15.44:0/1045018231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c740c7030 msgr2=0x7f2c74150a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:34.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.213+0000 7f2c78aac640 1 --2- 172.21.15.44:0/1045018231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c740c7030 0x7f2c74150a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:34.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.213+0000 7f2c78aac640 1 -- 172.21.15.44:0/1045018231 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c740c75e0 con 0x7f2c741552a0 2024-04-22T06:31:34.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.213+0000 7f2c79aae640 1 --2- 172.21.15.44:0/1045018231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c740c7030 0x7f2c74150a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:31:34.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.213+0000 7f2c792ad640 1 --2- 172.21.15.44:0/1045018231 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c740cc4e0 0x7f2c740c5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:31:34.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.213+0000 7f2c78aac640 1 --2- 172.21.15.44:0/1045018231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c741552a0 0x7f2c74153e50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2c6400c7f0 tx=0x7f2c6400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:34.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.214+0000 7f2c69ffb640 1 -- 172.21.15.44:0/1045018231 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c64015070 con 0x7f2c741552a0 2024-04-22T06:31:34.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.214+0000 7f2c69ffb640 1 -- 172.21.15.44:0/1045018231 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c640077c0 con 0x7f2c741552a0 2024-04-22T06:31:34.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.214+0000 7f2c7b511640 1 -- 172.21.15.44:0/1045018231 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c74158050 con 0x7f2c741552a0 2024-04-22T06:31:34.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.214+0000 7f2c69ffb640 1 -- 172.21.15.44:0/1045018231 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c64007b70 con 0x7f2c741552a0 2024-04-22T06:31:34.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.214+0000 7f2c7b511640 1 -- 172.21.15.44:0/1045018231 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c74158610 con 0x7f2c741552a0 2024-04-22T06:31:34.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.215+0000 7f2c7b511640 1 -- 172.21.15.44:0/1045018231 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c74068cc0 con 0x7f2c741552a0 2024-04-22T06:31:34.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.215+0000 7f2c69ffb640 1 -- 172.21.15.44:0/1045018231 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2c64018070 con 0x7f2c741552a0 2024-04-22T06:31:34.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.216+0000 7f2c69ffb640 1 --2- 172.21.15.44:0/1045018231 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f2c5806c8f0 0x7f2c5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:34.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.216+0000 7f2c69ffb640 1 -- 172.21.15.44:0/1045018231 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2c6408bee0 con 0x7f2c741552a0 2024-04-22T06:31:34.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.216+0000 7f2c792ad640 1 --2- 172.21.15.44:0/1045018231 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f2c5806c8f0 0x7f2c5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:34.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.216+0000 7f2c792ad640 1 --2- 172.21.15.44:0/1045018231 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f2c5806c8f0 0x7f2c5806edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2c70000f60 tx=0x7f2c70009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:34.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.220+0000 7f2c69ffb640 1 -- 172.21.15.44:0/1045018231 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2c6405a060 con 0x7f2c741552a0 2024-04-22T06:31:34.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.441+0000 7f2c7b511640 1 -- 172.21.15.44:0/1045018231 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c7415f4a0 con 0x7f2c741552a0 2024-04-22T06:31:34.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.441+0000 7f2c69ffb640 1 -- 172.21.15.44:0/1045018231 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+6838 (secure 0 0 0) 0x7f2c64059960 con 0x7f2c741552a0 2024-04-22T06:31:34.441 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:31:34.442 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":11,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/1779271860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":1779271860},{"type":"v1","addr":"172.21.15.70:6836","nonce":1779271860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1497302315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1497302315},{"type":"v1","addr":"172.21.15.44:6834","nonce":1497302315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/2383621014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2383621014},{"type":"v1","addr":"172.21.15.44:6835","nonce":2383621014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4293,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/915960587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":915960587},{"type":"v1","addr":"172.21.15.44:6837","nonce":915960587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T06:31:06.356312+0000","modified":"2024-04-22T06:31:33.449640+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1],"up":{"mds_0":4308,"mds_1":4305},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4305":{"gid":4305,"name":"b","rank":1,"incarnation":9,"state":"up:creating","state_seq":1,"addr":"172.21.15.70:6838/924955785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":924955785},{"type":"v1","addr":"172.21.15.70:6838","nonce":924955785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4308":{"gid":4308,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.70:6839/2915150040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6837","nonce":2915150040},{"type":"v1","addr":"172.21.15.70:6839","nonce":2915150040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4308,"qdb_cluster":[4308]},"id":1}]} 2024-04-22T06:31:34.442 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 11 2024-04-22T06:31:34.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.443+0000 7f2c7b511640 1 -- 172.21.15.44:0/1045018231 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f2c5806c8f0 msgr2=0x7f2c5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:34.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.443+0000 7f2c7b511640 1 --2- 172.21.15.44:0/1045018231 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f2c5806c8f0 0x7f2c5806edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2c70000f60 tx=0x7f2c70009040 comp rx=0 tx=0).stop 2024-04-22T06:31:34.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.443+0000 7f2c7b511640 1 -- 172.21.15.44:0/1045018231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c741552a0 msgr2=0x7f2c74153e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:34.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.443+0000 7f2c7b511640 1 --2- 172.21.15.44:0/1045018231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c741552a0 0x7f2c74153e50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2c6400c7f0 tx=0x7f2c6400ccb0 comp rx=0 tx=0).stop 2024-04-22T06:31:34.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.443+0000 7f2c7b511640 1 -- 172.21.15.44:0/1045018231 shutdown_connections 2024-04-22T06:31:34.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.443+0000 7f2c7b511640 1 --2- 172.21.15.44:0/1045018231 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f2c5806c8f0 0x7f2c5806edb0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:34.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.443+0000 7f2c7b511640 1 --2- 172.21.15.44:0/1045018231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c740c7030 0x7f2c74150a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:34.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.443+0000 7f2c7b511640 1 --2- 172.21.15.44:0/1045018231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c741552a0 0x7f2c74153e50 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:34.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.443+0000 7f2c7b511640 1 --2- 172.21.15.44:0/1045018231 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c740cc4e0 0x7f2c740c5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:34.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.443+0000 7f2c7b511640 1 -- 172.21.15.44:0/1045018231 >> 172.21.15.44:0/1045018231 conn(0x7f2c74075820 msgr2=0x7f2c74072a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:34.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.444+0000 7f2c7b511640 1 -- 172.21.15.44:0/1045018231 shutdown_connections 2024-04-22T06:31:34.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.444+0000 7f2c7b511640 1 -- 172.21.15.44:0/1045018231 wait complete. 2024-04-22T06:31:34.455 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T06:31:34.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.556+0000 7fe9936c1640 1 Processor -- start 2024-04-22T06:31:34.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.556+0000 7fe9936c1640 1 -- start start 2024-04-22T06:31:34.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.557+0000 7fe9936c1640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe98c155340 0x7fe98c0cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:34.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.557+0000 7fe9936c1640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe98c0cc310 0x7fe98c0cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:34.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.557+0000 7fe9936c1640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe98c0c5b70 con 0x7fe98c15e610 2024-04-22T06:31:34.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.557+0000 7fe9936c1640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fe98c15b020 con 0x7fe98c0cc310 2024-04-22T06:31:34.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.557+0000 7fe9936c1640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fe98c155720 con 0x7fe98c155340 2024-04-22T06:31:34.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.557+0000 7fe991c5e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe98c0cc310 0x7fe98c0cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:34.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.557+0000 7fe991c5e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe98c0cc310 0x7fe98c0cb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53368/0 (socket says 172.21.15.44:53368) 2024-04-22T06:31:34.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.557+0000 7fe991c5e640 1 -- 172.21.15.44:0/1719470714 learned_addr learned my addr 172.21.15.44:0/1719470714 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:34.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.557+0000 7fe99145d640 1 --2- 172.21.15.44:0/1719470714 >> v2:172.21.15.70:3300/0 conn(0x7fe98c155340 0x7fe98c0cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:34.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.557+0000 7fe991c5e640 1 -- 172.21.15.44:0/1719470714 >> v2:172.21.15.70:3300/0 conn(0x7fe98c155340 msgr2=0x7fe98c0cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:34.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.557+0000 7fe991c5e640 1 --2- 172.21.15.44:0/1719470714 >> v2:172.21.15.70:3300/0 conn(0x7fe98c155340 0x7fe98c0cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:34.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.557+0000 7fe991c5e640 1 -- 172.21.15.44:0/1719470714 >> v1:172.21.15.70:6789/0 conn(0x7fe98c15e610 legacy=0x7fe98c15b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:34.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.557+0000 7fe991c5e640 1 -- 172.21.15.44:0/1719470714 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe98c15eb50 con 0x7fe98c0cc310 2024-04-22T06:31:34.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.557+0000 7fe99145d640 1 --2- 172.21.15.44:0/1719470714 >> v2:172.21.15.70:3300/0 conn(0x7fe98c155340 0x7fe98c0cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:31:34.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.558+0000 7fe991c5e640 1 --2- 172.21.15.44:0/1719470714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe98c0cc310 0x7fe98c0cb500 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0ce190 tx=0x7fe98402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab5ce7b6137914f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:34.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.558+0000 7fe983fff640 1 -- 172.21.15.44:0/1719470714 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98402ca80 con 0x7fe98c0cc310 2024-04-22T06:31:34.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.558+0000 7fe983fff640 1 -- 172.21.15.44:0/1719470714 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe98402cc20 con 0x7fe98c0cc310 2024-04-22T06:31:34.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.558+0000 7fe983fff640 1 -- 172.21.15.44:0/1719470714 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9840343f0 con 0x7fe98c0cc310 2024-04-22T06:31:34.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.558+0000 7fe9936c1640 1 -- 172.21.15.44:0/1719470714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe98c0cc310 msgr2=0x7fe98c0cb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:34.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.558+0000 7fe9936c1640 1 --2- 172.21.15.44:0/1719470714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe98c0cc310 0x7fe98c0cb500 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0ce190 tx=0x7fe98402bee0 comp rx=0 tx=0).stop 2024-04-22T06:31:34.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.558+0000 7fe9936c1640 1 -- 172.21.15.44:0/1719470714 shutdown_connections 2024-04-22T06:31:34.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.558+0000 7fe9936c1640 1 --2- 172.21.15.44:0/1719470714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe98c0cc310 0x7fe98c0cb500 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:34.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.558+0000 7fe9936c1640 1 --2- 172.21.15.44:0/1719470714 >> v2:172.21.15.70:3300/0 conn(0x7fe98c155340 0x7fe98c0cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:34.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.558+0000 7fe9936c1640 1 -- 172.21.15.44:0/1719470714 >> 172.21.15.44:0/1719470714 conn(0x7fe98c06c560 msgr2=0x7fe98c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:34.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.559+0000 7fe9936c1640 1 -- 172.21.15.44:0/1719470714 shutdown_connections 2024-04-22T06:31:34.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.559+0000 7fe9936c1640 1 -- 172.21.15.44:0/1719470714 wait complete. 2024-04-22T06:31:34.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.560+0000 7fe9936c1640 1 Processor -- start 2024-04-22T06:31:34.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.560+0000 7fe9936c1640 1 -- start start 2024-04-22T06:31:34.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.560+0000 7fe9936c1640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe98c0cc310 0x7fe98c148be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:34.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.560+0000 7fe9936c1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe98c155340 0x7fe98c148200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:34.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.560+0000 7fe9936c1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe98c15e610 0x7fe98c147010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:34.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.560+0000 7fe9936c1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c15f3f0 con 0x7fe98c15e610 2024-04-22T06:31:34.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.560+0000 7fe9936c1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c15f110 con 0x7fe98c155340 2024-04-22T06:31:34.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.560+0000 7fe9936c1640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe98c15b2e0 con 0x7fe98c0cc310 2024-04-22T06:31:34.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.560+0000 7fe991c5e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe98c15e610 0x7fe98c147010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:34.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.560+0000 7fe991c5e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe98c15e610 0x7fe98c147010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53380/0 (socket says 172.21.15.44:53380) 2024-04-22T06:31:34.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.561+0000 7fe991c5e640 1 -- 172.21.15.44:0/3384741073 learned_addr learned my addr 172.21.15.44:0/3384741073 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:34.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.561+0000 7fe99145d640 1 --2- 172.21.15.44:0/3384741073 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe98c0cc310 0x7fe98c148be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:34.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.561+0000 7fe990c5c640 1 --2- 172.21.15.44:0/3384741073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe98c155340 0x7fe98c148200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:34.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.561+0000 7fe991c5e640 1 -- 172.21.15.44:0/3384741073 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe98c0cc310 msgr2=0x7fe98c148be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:34.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.561+0000 7fe991c5e640 1 --2- 172.21.15.44:0/3384741073 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe98c0cc310 0x7fe98c148be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:34.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.561+0000 7fe991c5e640 1 -- 172.21.15.44:0/3384741073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe98c155340 msgr2=0x7fe98c148200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:34.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.561+0000 7fe991c5e640 1 --2- 172.21.15.44:0/3384741073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe98c155340 0x7fe98c148200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:34.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.561+0000 7fe991c5e640 1 -- 172.21.15.44:0/3384741073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe98c149120 con 0x7fe98c15e610 2024-04-22T06:31:34.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.561+0000 7fe991c5e640 1 --2- 172.21.15.44:0/3384741073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe98c15e610 0x7fe98c147010 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe984009f40 tx=0x7fe984004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:34.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.561+0000 7fe981ffb640 1 -- 172.21.15.44:0/3384741073 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe984043020 con 0x7fe98c15e610 2024-04-22T06:31:34.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.561+0000 7fe981ffb640 1 -- 172.21.15.44:0/3384741073 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe98403e450 con 0x7fe98c15e610 2024-04-22T06:31:34.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.561+0000 7fe981ffb640 1 -- 172.21.15.44:0/3384741073 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe984009510 con 0x7fe98c15e610 2024-04-22T06:31:34.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.561+0000 7fe9936c1640 1 -- 172.21.15.44:0/3384741073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe98c15cd30 con 0x7fe98c15e610 2024-04-22T06:31:34.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.561+0000 7fe9936c1640 1 -- 172.21.15.44:0/3384741073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe98c15d2f0 con 0x7fe98c15e610 2024-04-22T06:31:34.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.562+0000 7fe9936c1640 1 -- 172.21.15.44:0/3384741073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe98c068cc0 con 0x7fe98c15e610 2024-04-22T06:31:34.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.563+0000 7fe981ffb640 1 -- 172.21.15.44:0/3384741073 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe984046020 con 0x7fe98c15e610 2024-04-22T06:31:34.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.563+0000 7fe981ffb640 1 --2- 172.21.15.44:0/3384741073 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fe97006c8f0 0x7fe97006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:34.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.563+0000 7fe981ffb640 1 -- 172.21.15.44:0/3384741073 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe9840ae660 con 0x7fe98c15e610 2024-04-22T06:31:34.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.563+0000 7fe99145d640 1 --2- 172.21.15.44:0/3384741073 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fe97006c8f0 0x7fe97006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:34.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.564+0000 7fe99145d640 1 --2- 172.21.15.44:0/3384741073 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fe97006c8f0 0x7fe97006edb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe98800ab60 tx=0x7fe988008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:34.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.568+0000 7fe981ffb640 1 -- 172.21.15.44:0/3384741073 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe98404d030 con 0x7fe98c15e610 2024-04-22T06:31:34.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.754+0000 7fe9936c1640 1 -- 172.21.15.44:0/3384741073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe98c0cc040 con 0x7fe98c15e610 2024-04-22T06:31:34.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.755+0000 7fe981ffb640 1 -- 172.21.15.44:0/3384741073 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18466 (secure 0 0 0) 0x7fe9840ab840 con 0x7fe98c15e610 2024-04-22T06:31:34.755 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:31:34.755 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"fsid":"9b0a34ca-403d-4513-8a66-d3749ba559aa","created":"2024-04-22T06:30:42.334581+0000","modified":"2024-04-22T06:31:11.406256+0000","last_up_change":"2024-04-22T06:30:52.942257+0000","last_in_change":"2024-04-22T06:30:44.467727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T06:30:54.560256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T06:31:04.612387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T06:31:05.312124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-22T06:31:09.025550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b4a6692-53c9-472d-b2f3-1dd29a6679ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6817","nonce":2220268493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6819","nonce":2220268493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6823","nonce":2220268493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6821","nonce":2220268493}]},"public_addr":"172.21.15.44:6817/2220268493","cluster_addr":"172.21.15.44:6819/2220268493","heartbeat_back_addr":"172.21.15.44:6823/2220268493","heartbeat_front_addr":"172.21.15.44:6821/2220268493","state":["exists","up"]},{"osd":1,"uuid":"70a6370a-7c0a-4d64-a0e4-b2e726b4f291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6809","nonce":1278678977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6811","nonce":1278678977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6815","nonce":1278678977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6813","nonce":1278678977}]},"public_addr":"172.21.15.44:6809/1278678977","cluster_addr":"172.21.15.44:6811/1278678977","heartbeat_back_addr":"172.21.15.44:6815/1278678977","heartbeat_front_addr":"172.21.15.44:6813/1278678977","state":["exists","up"]},{"osd":2,"uuid":"f2e51a62-ca91-470d-bd5a-5710c61deee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6801","nonce":2544087903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6803","nonce":2544087903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6807","nonce":2544087903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6805","nonce":2544087903}]},"public_addr":"172.21.15.44:6801/2544087903","cluster_addr":"172.21.15.44:6803/2544087903","heartbeat_back_addr":"172.21.15.44:6807/2544087903","heartbeat_front_addr":"172.21.15.44:6805/2544087903","state":["exists","up"]},{"osd":3,"uuid":"bc514203-9af9-4f1f-b18e-765bdd5de6e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6825","nonce":1692928109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6827","nonce":1692928109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6831","nonce":1692928109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6829","nonce":1692928109}]},"public_addr":"172.21.15.44:6825/1692928109","cluster_addr":"172.21.15.44:6827/1692928109","heartbeat_back_addr":"172.21.15.44:6831/1692928109","heartbeat_front_addr":"172.21.15.44:6829/1692928109","state":["exists","up"]},{"osd":4,"uuid":"357c439a-6e08-4c19-ac18-b68e6c890497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6820","nonce":3802577427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6824","nonce":3802577427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6831","nonce":3802577427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6828","nonce":3802577427}]},"public_addr":"172.21.15.70:6820/3802577427","cluster_addr":"172.21.15.70:6824/3802577427","heartbeat_back_addr":"172.21.15.70:6831/3802577427","heartbeat_front_addr":"172.21.15.70:6828/3802577427","state":["exists","up"]},{"osd":5,"uuid":"952fb88f-d56e-4305-bfa8-d32674332dfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6801","nonce":664757345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6803","nonce":664757345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6807","nonce":664757345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6805","nonce":664757345}]},"public_addr":"172.21.15.70:6801/664757345","cluster_addr":"172.21.15.70:6803/664757345","heartbeat_back_addr":"172.21.15.70:6807/664757345","heartbeat_front_addr":"172.21.15.70:6805/664757345","state":["exists","up"]},{"osd":6,"uuid":"74399b3d-fac2-45ab-9172-2cd95b02450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6817","nonce":1623253482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6821","nonce":1623253482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6827","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6829","nonce":1623253482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6825","nonce":1623253482}]},"public_addr":"172.21.15.70:6817/1623253482","cluster_addr":"172.21.15.70:6821/1623253482","heartbeat_back_addr":"172.21.15.70:6829/1623253482","heartbeat_front_addr":"172.21.15.70:6825/1623253482","state":["exists","up"]},{"osd":7,"uuid":"8f35508d-6fcc-43ff-8ee7-0498a7ab3b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6809","nonce":1551180300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6811","nonce":1551180300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6815","nonce":1551180300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6813","nonce":1551180300}]},"public_addr":"172.21.15.70:6809/1551180300","cluster_addr":"172.21.15.70:6811/1551180300","heartbeat_back_addr":"172.21.15.70:6815/1551180300","heartbeat_front_addr":"172.21.15.70:6813/1551180300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.612502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.653756+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.521169+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.649955+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.640231+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.686638+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.642199+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.700360+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T06:31:34.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.757+0000 7fe9936c1640 1 -- 172.21.15.44:0/3384741073 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fe97006c8f0 msgr2=0x7fe97006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:34.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.757+0000 7fe9936c1640 1 --2- 172.21.15.44:0/3384741073 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fe97006c8f0 0x7fe97006edb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe98800ab60 tx=0x7fe988008040 comp rx=0 tx=0).stop 2024-04-22T06:31:34.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.757+0000 7fe9936c1640 1 -- 172.21.15.44:0/3384741073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe98c15e610 msgr2=0x7fe98c147010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:34.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.757+0000 7fe9936c1640 1 --2- 172.21.15.44:0/3384741073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe98c15e610 0x7fe98c147010 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe984009f40 tx=0x7fe984004a40 comp rx=0 tx=0).stop 2024-04-22T06:31:34.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.757+0000 7fe9936c1640 1 -- 172.21.15.44:0/3384741073 shutdown_connections 2024-04-22T06:31:34.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.757+0000 7fe9936c1640 1 --2- 172.21.15.44:0/3384741073 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fe97006c8f0 0x7fe97006edb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:34.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.757+0000 7fe9936c1640 1 --2- 172.21.15.44:0/3384741073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe98c15e610 0x7fe98c147010 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:34.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.757+0000 7fe9936c1640 1 --2- 172.21.15.44:0/3384741073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe98c155340 0x7fe98c148200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:34.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.757+0000 7fe9936c1640 1 --2- 172.21.15.44:0/3384741073 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe98c0cc310 0x7fe98c148be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:34.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.757+0000 7fe9936c1640 1 -- 172.21.15.44:0/3384741073 >> 172.21.15.44:0/3384741073 conn(0x7fe98c06c560 msgr2=0x7fe98c14a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:34.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.757+0000 7fe9936c1640 1 -- 172.21.15.44:0/3384741073 shutdown_connections 2024-04-22T06:31:34.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.757+0000 7fe9936c1640 1 -- 172.21.15.44:0/3384741073 wait complete. 2024-04-22T06:31:34.768 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-22T06:31:34.768 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T06:31:34.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.868+0000 7f117fec1640 1 Processor -- start 2024-04-22T06:31:34.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.869+0000 7f117fec1640 1 -- start start 2024-04-22T06:31:34.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.869+0000 7f117fec1640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1178154f40 0x7f1178073a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:34.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.869+0000 7f117fec1640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1178074260 0x7f117815ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:34.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.869+0000 7f117fec1640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f117806c560 con 0x7f1178155320 2024-04-22T06:31:34.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.869+0000 7f117fec1640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f117815bfa0 con 0x7f1178154f40 2024-04-22T06:31:34.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.869+0000 7f117fec1640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f117815b420 con 0x7f1178074260 2024-04-22T06:31:34.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.870+0000 7f117dc5d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1178154f40 0x7f1178073a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:34.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.870+0000 7f117dc5d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1178154f40 0x7f1178073a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53392/0 (socket says 172.21.15.44:53392) 2024-04-22T06:31:34.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.870+0000 7f117dc5d640 1 -- 172.21.15.44:0/119414276 learned_addr learned my addr 172.21.15.44:0/119414276 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:34.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.870+0000 7f117e45e640 1 --2- 172.21.15.44:0/119414276 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1178074260 0x7f117815ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:34.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.870+0000 7f117dc5d640 1 -- 172.21.15.44:0/119414276 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1178074260 msgr2=0x7f117815ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:34.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.870+0000 7f117dc5d640 1 --2- 172.21.15.44:0/119414276 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1178074260 0x7f117815ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:34.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.870+0000 7f117dc5d640 1 -- 172.21.15.44:0/119414276 >> v1:172.21.15.70:6789/0 conn(0x7f1178155320 legacy=0x7f11780ce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:34.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.870+0000 7f117dc5d640 1 -- 172.21.15.44:0/119414276 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f117815b700 con 0x7f1178154f40 2024-04-22T06:31:34.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.870+0000 7f117e45e640 1 --2- 172.21.15.44:0/119414276 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1178074260 0x7f117815ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:31:34.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.871+0000 7f117dc5d640 1 --2- 172.21.15.44:0/119414276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1178154f40 0x7f1178073a10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1174009870 tx=0x7f117402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1726e663995d4df server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:34.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.871+0000 7f117cc5b640 1 -- 172.21.15.44:0/119414276 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117402f840 con 0x7f1178154f40 2024-04-22T06:31:34.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.871+0000 7f117cc5b640 1 -- 172.21.15.44:0/119414276 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f117402f9e0 con 0x7f1178154f40 2024-04-22T06:31:34.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.871+0000 7f117cc5b640 1 -- 172.21.15.44:0/119414276 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117402fdb0 con 0x7f1178154f40 2024-04-22T06:31:34.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.871+0000 7f117fec1640 1 -- 172.21.15.44:0/119414276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1178154f40 msgr2=0x7f1178073a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:34.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.871+0000 7f117fec1640 1 --2- 172.21.15.44:0/119414276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1178154f40 0x7f1178073a10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1174009870 tx=0x7f117402ba20 comp rx=0 tx=0).stop 2024-04-22T06:31:34.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.871+0000 7f117fec1640 1 -- 172.21.15.44:0/119414276 shutdown_connections 2024-04-22T06:31:34.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.871+0000 7f117fec1640 1 --2- 172.21.15.44:0/119414276 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1178074260 0x7f117815ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:34.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.871+0000 7f117fec1640 1 --2- 172.21.15.44:0/119414276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1178154f40 0x7f1178073a10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:34.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.871+0000 7f117fec1640 1 -- 172.21.15.44:0/119414276 >> 172.21.15.44:0/119414276 conn(0x7f1178074cb0 msgr2=0x7f11780750c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:34.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.871+0000 7f117fec1640 1 -- 172.21.15.44:0/119414276 shutdown_connections 2024-04-22T06:31:34.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.871+0000 7f117fec1640 1 -- 172.21.15.44:0/119414276 wait complete. 2024-04-22T06:31:34.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.873+0000 7f117fec1640 1 Processor -- start 2024-04-22T06:31:34.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.873+0000 7f117fec1640 1 -- start start 2024-04-22T06:31:34.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.873+0000 7f117fec1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1178074260 0x7f11780c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:34.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.873+0000 7f117fec1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1178154f40 0x7f1178156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:34.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.873+0000 7f117fec1640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11781f1a30 0x7f11781f4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:34.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.873+0000 7f117fec1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f117815e6e0 con 0x7f1178154f40 2024-04-22T06:31:34.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.873+0000 7f117fec1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f117815bcc0 con 0x7f1178074260 2024-04-22T06:31:34.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.873+0000 7f117fec1640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f1178073f50 con 0x7f11781f1a30 2024-04-22T06:31:34.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.873+0000 7f117d45c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1178154f40 0x7f1178156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:34.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.873+0000 7f117d45c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1178154f40 0x7f1178156570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53400/0 (socket says 172.21.15.44:53400) 2024-04-22T06:31:34.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.873+0000 7f117d45c640 1 -- 172.21.15.44:0/2105513188 learned_addr learned my addr 172.21.15.44:0/2105513188 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:34.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.874+0000 7f117e45e640 1 --2- 172.21.15.44:0/2105513188 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11781f1a30 0x7f11781f4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:34.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.874+0000 7f117dc5d640 1 --2- 172.21.15.44:0/2105513188 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1178074260 0x7f11780c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:34.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.874+0000 7f117d45c640 1 -- 172.21.15.44:0/2105513188 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11781f1a30 msgr2=0x7f11781f4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:34.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.874+0000 7f117d45c640 1 --2- 172.21.15.44:0/2105513188 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11781f1a30 0x7f11781f4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:34.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.874+0000 7f117d45c640 1 -- 172.21.15.44:0/2105513188 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1178074260 msgr2=0x7f11780c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:34.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.874+0000 7f117d45c640 1 --2- 172.21.15.44:0/2105513188 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1178074260 0x7f11780c89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:34.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.874+0000 7f117d45c640 1 -- 172.21.15.44:0/2105513188 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11781f4860 con 0x7f1178154f40 2024-04-22T06:31:34.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.874+0000 7f117d45c640 1 --2- 172.21.15.44:0/2105513188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1178154f40 0x7f1178156570 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f116800c340 tx=0x7f116800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:34.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.874+0000 7f116e7fc640 1 -- 172.21.15.44:0/2105513188 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1168017070 con 0x7f1178154f40 2024-04-22T06:31:34.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.874+0000 7f116e7fc640 1 -- 172.21.15.44:0/2105513188 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1168005310 con 0x7f1178154f40 2024-04-22T06:31:34.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.874+0000 7f116e7fc640 1 -- 172.21.15.44:0/2105513188 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1168003850 con 0x7f1178154f40 2024-04-22T06:31:34.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.874+0000 7f117fec1640 1 -- 172.21.15.44:0/2105513188 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1178156f90 con 0x7f1178154f40 2024-04-22T06:31:34.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.874+0000 7f117fec1640 1 -- 172.21.15.44:0/2105513188 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11781f0f50 con 0x7f1178154f40 2024-04-22T06:31:34.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.875+0000 7f117fec1640 1 -- 172.21.15.44:0/2105513188 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1178069310 con 0x7f1178154f40 2024-04-22T06:31:34.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.875+0000 7f116e7fc640 1 -- 172.21.15.44:0/2105513188 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f116801a070 con 0x7f1178154f40 2024-04-22T06:31:34.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.876+0000 7f116e7fc640 1 --2- 172.21.15.44:0/2105513188 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f115c06c920 0x7f115c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:34.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.876+0000 7f116e7fc640 1 -- 172.21.15.44:0/2105513188 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f116808b040 con 0x7f1178154f40 2024-04-22T06:31:34.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.876+0000 7f117dc5d640 1 --2- 172.21.15.44:0/2105513188 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f115c06c920 0x7f115c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:34.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.877+0000 7f117dc5d640 1 --2- 172.21.15.44:0/2105513188 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f115c06c920 0x7f115c06ede0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1174036000 tx=0x7f1174008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:34.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:34.880+0000 7f116e7fc640 1 -- 172.21.15.44:0/2105513188 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1168058a50 con 0x7f1178154f40 2024-04-22T06:31:35.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.067+0000 7f117fec1640 1 -- 172.21.15.44:0/2105513188 --> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1178157550 con 0x7f115c06c920 2024-04-22T06:31:35.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.076+0000 7f116e7fc640 1 -- 172.21.15.44:0/2105513188 <== mgr.4111 v2:172.21.15.70:6832/2570466131 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f1178157550 con 0x7f115c06c920 2024-04-22T06:31:35.080 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:31:35.082 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-04-22T06:31:35.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.079+0000 7f117fec1640 1 -- 172.21.15.44:0/2105513188 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f115c06c920 msgr2=0x7f115c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:35.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.079+0000 7f117fec1640 1 --2- 172.21.15.44:0/2105513188 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f115c06c920 0x7f115c06ede0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1174036000 tx=0x7f1174008030 comp rx=0 tx=0).stop 2024-04-22T06:31:35.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.079+0000 7f117fec1640 1 -- 172.21.15.44:0/2105513188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1178154f40 msgr2=0x7f1178156570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:35.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.079+0000 7f117fec1640 1 --2- 172.21.15.44:0/2105513188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1178154f40 0x7f1178156570 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f116800c340 tx=0x7f116800c800 comp rx=0 tx=0).stop 2024-04-22T06:31:35.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.079+0000 7f117fec1640 1 -- 172.21.15.44:0/2105513188 shutdown_connections 2024-04-22T06:31:35.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.079+0000 7f117fec1640 1 --2- 172.21.15.44:0/2105513188 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f115c06c920 0x7f115c06ede0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:35.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.079+0000 7f117fec1640 1 --2- 172.21.15.44:0/2105513188 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11781f1a30 0x7f11781f4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:35.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.079+0000 7f117fec1640 1 --2- 172.21.15.44:0/2105513188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1178154f40 0x7f1178156570 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:35.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.079+0000 7f117fec1640 1 --2- 172.21.15.44:0/2105513188 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1178074260 0x7f11780c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:35.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.079+0000 7f117fec1640 1 -- 172.21.15.44:0/2105513188 >> 172.21.15.44:0/2105513188 conn(0x7f1178074cb0 msgr2=0x7f11781e43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:35.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.079+0000 7f117fec1640 1 -- 172.21.15.44:0/2105513188 shutdown_connections 2024-04-22T06:31:35.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.079+0000 7f117fec1640 1 -- 172.21.15.44:0/2105513188 wait complete. 2024-04-22T06:31:35.090 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-04-22T06:31:34.551837+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236148,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738572,"statfs":{"total":773094113280,"available":772852297728,"internally_reserved":0,"allocated":3383296,"data_stored":2029432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002999"},"pg_stats":[{"pgid":"2.1f","version":"24'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560936+0000","last_change":"2024-04-22T06:31:06.002169+0000","last_active":"2024-04-22T06:31:12.560936+0000","last_peered":"2024-04-22T06:31:12.560936+0000","last_clean":"2024-04-22T06:31:12.560936+0000","last_became_active":"2024-04-22T06:31:06.001899+0000","last_became_peered":"2024-04-22T06:31:06.001899+0000","last_unstale":"2024-04-22T06:31:12.560936+0000","last_undegraded":"2024-04-22T06:31:12.560936+0000","last_fullsized":"2024-04-22T06:31:12.560936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:11:48.056404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559298+0000","last_change":"2024-04-22T06:31:07.025982+0000","last_active":"2024-04-22T06:31:12.559298+0000","last_peered":"2024-04-22T06:31:12.559298+0000","last_clean":"2024-04-22T06:31:12.559298+0000","last_became_active":"2024-04-22T06:31:07.025650+0000","last_became_peered":"2024-04-22T06:31:07.025650+0000","last_unstale":"2024-04-22T06:31:12.559298+0000","last_undegraded":"2024-04-22T06:31:12.559298+0000","last_fullsized":"2024-04-22T06:31:12.559298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:58:58.029898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560389+0000","last_change":"2024-04-22T06:31:11.048083+0000","last_active":"2024-04-22T06:31:12.560389+0000","last_peered":"2024-04-22T06:31:12.560389+0000","last_clean":"2024-04-22T06:31:12.560389+0000","last_became_active":"2024-04-22T06:31:11.047682+0000","last_became_peered":"2024-04-22T06:31:11.047682+0000","last_unstale":"2024-04-22T06:31:12.560389+0000","last_undegraded":"2024-04-22T06:31:12.560389+0000","last_fullsized":"2024-04-22T06:31:12.560389+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:34:30.158935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"24'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559543+0000","last_change":"2024-04-22T06:31:06.007484+0000","last_active":"2024-04-22T06:31:12.559543+0000","last_peered":"2024-04-22T06:31:12.559543+0000","last_clean":"2024-04-22T06:31:12.559543+0000","last_became_active":"2024-04-22T06:31:06.007204+0000","last_became_peered":"2024-04-22T06:31:06.007204+0000","last_unstale":"2024-04-22T06:31:12.559543+0000","last_undegraded":"2024-04-22T06:31:12.559543+0000","last_fullsized":"2024-04-22T06:31:12.559543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:39:18.905813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560596+0000","last_change":"2024-04-22T06:31:07.014658+0000","last_active":"2024-04-22T06:31:12.560596+0000","last_peered":"2024-04-22T06:31:12.560596+0000","last_clean":"2024-04-22T06:31:12.560596+0000","last_became_active":"2024-04-22T06:31:07.014283+0000","last_became_peered":"2024-04-22T06:31:07.014283+0000","last_unstale":"2024-04-22T06:31:12.560596+0000","last_undegraded":"2024-04-22T06:31:12.560596+0000","last_fullsized":"2024-04-22T06:31:12.560596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:43:01.159439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.032692+0000","last_change":"2024-04-22T06:31:11.045114+0000","last_active":"2024-04-22T06:31:12.032692+0000","last_peered":"2024-04-22T06:31:12.032692+0000","last_clean":"2024-04-22T06:31:12.032692+0000","last_became_active":"2024-04-22T06:31:11.044728+0000","last_became_peered":"2024-04-22T06:31:11.044728+0000","last_unstale":"2024-04-22T06:31:12.032692+0000","last_undegraded":"2024-04-22T06:31:12.032692+0000","last_fullsized":"2024-04-22T06:31:12.032692+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:48:19.215184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"24'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031494+0000","last_change":"2024-04-22T06:31:05.995468+0000","last_active":"2024-04-22T06:31:12.031494+0000","last_peered":"2024-04-22T06:31:12.031494+0000","last_clean":"2024-04-22T06:31:12.031494+0000","last_became_active":"2024-04-22T06:31:05.995159+0000","last_became_peered":"2024-04-22T06:31:05.995159+0000","last_unstale":"2024-04-22T06:31:12.031494+0000","last_undegraded":"2024-04-22T06:31:12.031494+0000","last_fullsized":"2024-04-22T06:31:12.031494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:01:12.223718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031966+0000","last_change":"2024-04-22T06:31:07.020024+0000","last_active":"2024-04-22T06:31:12.031966+0000","last_peered":"2024-04-22T06:31:12.031966+0000","last_clean":"2024-04-22T06:31:12.031966+0000","last_became_active":"2024-04-22T06:31:07.019490+0000","last_became_peered":"2024-04-22T06:31:07.019490+0000","last_unstale":"2024-04-22T06:31:12.031966+0000","last_undegraded":"2024-04-22T06:31:12.031966+0000","last_fullsized":"2024-04-22T06:31:12.031966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:26:41.811978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.033365+0000","last_change":"2024-04-22T06:31:11.057556+0000","last_active":"2024-04-22T06:31:12.033365+0000","last_peered":"2024-04-22T06:31:12.033365+0000","last_clean":"2024-04-22T06:31:12.033365+0000","last_became_active":"2024-04-22T06:31:11.057174+0000","last_became_peered":"2024-04-22T06:31:11.057174+0000","last_unstale":"2024-04-22T06:31:12.033365+0000","last_undegraded":"2024-04-22T06:31:12.033365+0000","last_fullsized":"2024-04-22T06:31:12.033365+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:41:19.526548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030542+0000","last_change":"2024-04-22T06:31:05.996281+0000","last_active":"2024-04-22T06:31:12.030542+0000","last_peered":"2024-04-22T06:31:12.030542+0000","last_clean":"2024-04-22T06:31:12.030542+0000","last_became_active":"2024-04-22T06:31:05.995951+0000","last_became_peered":"2024-04-22T06:31:05.995951+0000","last_unstale":"2024-04-22T06:31:12.030542+0000","last_undegraded":"2024-04-22T06:31:12.030542+0000","last_fullsized":"2024-04-22T06:31:12.030542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:01:43.732947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.032029+0000","last_change":"2024-04-22T06:31:07.017755+0000","last_active":"2024-04-22T06:31:12.032029+0000","last_peered":"2024-04-22T06:31:12.032029+0000","last_clean":"2024-04-22T06:31:12.032029+0000","last_became_active":"2024-04-22T06:31:07.017420+0000","last_became_peered":"2024-04-22T06:31:07.017420+0000","last_unstale":"2024-04-22T06:31:12.032029+0000","last_undegraded":"2024-04-22T06:31:12.032029+0000","last_fullsized":"2024-04-22T06:31:12.032029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:30:57.913020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.032802+0000","last_change":"2024-04-22T06:31:11.049497+0000","last_active":"2024-04-22T06:31:12.032802+0000","last_peered":"2024-04-22T06:31:12.032802+0000","last_clean":"2024-04-22T06:31:12.032802+0000","last_became_active":"2024-04-22T06:31:11.049079+0000","last_became_peered":"2024-04-22T06:31:11.049079+0000","last_unstale":"2024-04-22T06:31:12.032802+0000","last_undegraded":"2024-04-22T06:31:12.032802+0000","last_fullsized":"2024-04-22T06:31:12.032802+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:29:02.299691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558561+0000","last_change":"2024-04-22T06:31:07.023043+0000","last_active":"2024-04-22T06:31:12.558561+0000","last_peered":"2024-04-22T06:31:12.558561+0000","last_clean":"2024-04-22T06:31:12.558561+0000","last_became_active":"2024-04-22T06:31:07.022747+0000","last_became_peered":"2024-04-22T06:31:07.022747+0000","last_unstale":"2024-04-22T06:31:12.558561+0000","last_undegraded":"2024-04-22T06:31:12.558561+0000","last_fullsized":"2024-04-22T06:31:12.558561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:52:33.626072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"24'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030958+0000","last_change":"2024-04-22T06:31:05.997936+0000","last_active":"2024-04-22T06:31:12.030958+0000","last_peered":"2024-04-22T06:31:12.030958+0000","last_clean":"2024-04-22T06:31:12.030958+0000","last_became_active":"2024-04-22T06:31:05.997618+0000","last_became_peered":"2024-04-22T06:31:05.997618+0000","last_unstale":"2024-04-22T06:31:12.030958+0000","last_undegraded":"2024-04-22T06:31:12.030958+0000","last_fullsized":"2024-04-22T06:31:12.030958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:43:00.396691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.033550+0000","last_change":"2024-04-22T06:31:11.047725+0000","last_active":"2024-04-22T06:31:12.033550+0000","last_peered":"2024-04-22T06:31:12.033550+0000","last_clean":"2024-04-22T06:31:12.033550+0000","last_became_active":"2024-04-22T06:31:11.047278+0000","last_became_peered":"2024-04-22T06:31:11.047278+0000","last_unstale":"2024-04-22T06:31:12.033550+0000","last_undegraded":"2024-04-22T06:31:12.033550+0000","last_fullsized":"2024-04-22T06:31:12.033550+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:32:56.201059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561171+0000","last_change":"2024-04-22T06:31:07.021250+0000","last_active":"2024-04-22T06:31:12.561171+0000","last_peered":"2024-04-22T06:31:12.561171+0000","last_clean":"2024-04-22T06:31:12.561171+0000","last_became_active":"2024-04-22T06:31:07.020876+0000","last_became_peered":"2024-04-22T06:31:07.020876+0000","last_unstale":"2024-04-22T06:31:12.561171+0000","last_undegraded":"2024-04-22T06:31:12.561171+0000","last_fullsized":"2024-04-22T06:31:12.561171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:21:31.210819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561332+0000","last_change":"2024-04-22T06:31:06.012775+0000","last_active":"2024-04-22T06:31:12.561332+0000","last_peered":"2024-04-22T06:31:12.561332+0000","last_clean":"2024-04-22T06:31:12.561332+0000","last_became_active":"2024-04-22T06:31:06.012474+0000","last_became_peered":"2024-04-22T06:31:06.012474+0000","last_unstale":"2024-04-22T06:31:12.561332+0000","last_undegraded":"2024-04-22T06:31:12.561332+0000","last_fullsized":"2024-04-22T06:31:12.561332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:09:48.197011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.033219+0000","last_change":"2024-04-22T06:31:11.057641+0000","last_active":"2024-04-22T06:31:12.033219+0000","last_peered":"2024-04-22T06:31:12.033219+0000","last_clean":"2024-04-22T06:31:12.033219+0000","last_became_active":"2024-04-22T06:31:11.057221+0000","last_became_peered":"2024-04-22T06:31:11.057221+0000","last_unstale":"2024-04-22T06:31:12.033219+0000","last_undegraded":"2024-04-22T06:31:12.033219+0000","last_fullsized":"2024-04-22T06:31:12.033219+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:14:57.713113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559730+0000","last_change":"2024-04-22T06:31:07.020822+0000","last_active":"2024-04-22T06:31:12.559730+0000","last_peered":"2024-04-22T06:31:12.559730+0000","last_clean":"2024-04-22T06:31:12.559730+0000","last_became_active":"2024-04-22T06:31:07.020484+0000","last_became_peered":"2024-04-22T06:31:07.020484+0000","last_unstale":"2024-04-22T06:31:12.559730+0000","last_undegraded":"2024-04-22T06:31:12.559730+0000","last_fullsized":"2024-04-22T06:31:12.559730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:18:26.522131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560581+0000","last_change":"2024-04-22T06:31:06.000000+0000","last_active":"2024-04-22T06:31:12.560581+0000","last_peered":"2024-04-22T06:31:12.560581+0000","last_clean":"2024-04-22T06:31:12.560581+0000","last_became_active":"2024-04-22T06:31:05.999663+0000","last_became_peered":"2024-04-22T06:31:05.999663+0000","last_unstale":"2024-04-22T06:31:12.560581+0000","last_undegraded":"2024-04-22T06:31:12.560581+0000","last_fullsized":"2024-04-22T06:31:12.560581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:42.805084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560901+0000","last_change":"2024-04-22T06:31:11.057991+0000","last_active":"2024-04-22T06:31:12.560901+0000","last_peered":"2024-04-22T06:31:12.560901+0000","last_clean":"2024-04-22T06:31:12.560901+0000","last_became_active":"2024-04-22T06:31:11.057595+0000","last_became_peered":"2024-04-22T06:31:11.057595+0000","last_unstale":"2024-04-22T06:31:12.560901+0000","last_undegraded":"2024-04-22T06:31:12.560901+0000","last_fullsized":"2024-04-22T06:31:12.560901+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:17:06.366663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.033006+0000","last_change":"2024-04-22T06:31:07.017303+0000","last_active":"2024-04-22T06:31:12.033006+0000","last_peered":"2024-04-22T06:31:12.033006+0000","last_clean":"2024-04-22T06:31:12.033006+0000","last_became_active":"2024-04-22T06:31:07.016990+0000","last_became_peered":"2024-04-22T06:31:07.016990+0000","last_unstale":"2024-04-22T06:31:12.033006+0000","last_undegraded":"2024-04-22T06:31:12.033006+0000","last_fullsized":"2024-04-22T06:31:12.033006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:51:38.275472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.029843+0000","last_change":"2024-04-22T06:31:05.997363+0000","last_active":"2024-04-22T06:31:12.029843+0000","last_peered":"2024-04-22T06:31:12.029843+0000","last_clean":"2024-04-22T06:31:12.029843+0000","last_became_active":"2024-04-22T06:31:05.996982+0000","last_became_peered":"2024-04-22T06:31:05.996982+0000","last_unstale":"2024-04-22T06:31:12.029843+0000","last_undegraded":"2024-04-22T06:31:12.029843+0000","last_fullsized":"2024-04-22T06:31:12.029843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:23:50.052130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.033517+0000","last_change":"2024-04-22T06:31:11.048507+0000","last_active":"2024-04-22T06:31:12.033517+0000","last_peered":"2024-04-22T06:31:12.033517+0000","last_clean":"2024-04-22T06:31:12.033517+0000","last_became_active":"2024-04-22T06:31:11.047722+0000","last_became_peered":"2024-04-22T06:31:11.047722+0000","last_unstale":"2024-04-22T06:31:12.033517+0000","last_undegraded":"2024-04-22T06:31:12.033517+0000","last_fullsized":"2024-04-22T06:31:12.033517+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:04:48.732029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561139+0000","last_change":"2024-04-22T06:31:07.021095+0000","last_active":"2024-04-22T06:31:12.561139+0000","last_peered":"2024-04-22T06:31:12.561139+0000","last_clean":"2024-04-22T06:31:12.561139+0000","last_became_active":"2024-04-22T06:31:07.020814+0000","last_became_peered":"2024-04-22T06:31:07.020814+0000","last_unstale":"2024-04-22T06:31:12.561139+0000","last_undegraded":"2024-04-22T06:31:12.561139+0000","last_fullsized":"2024-04-22T06:31:12.561139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:27:08.869392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"24'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559162+0000","last_change":"2024-04-22T06:31:05.997508+0000","last_active":"2024-04-22T06:31:12.559162+0000","last_peered":"2024-04-22T06:31:12.559162+0000","last_clean":"2024-04-22T06:31:12.559162+0000","last_became_active":"2024-04-22T06:31:05.997233+0000","last_became_peered":"2024-04-22T06:31:05.997233+0000","last_unstale":"2024-04-22T06:31:12.559162+0000","last_undegraded":"2024-04-22T06:31:12.559162+0000","last_fullsized":"2024-04-22T06:31:12.559162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:40:56.909549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.032801+0000","last_change":"2024-04-22T06:31:11.045424+0000","last_active":"2024-04-22T06:31:12.032801+0000","last_peered":"2024-04-22T06:31:12.032801+0000","last_clean":"2024-04-22T06:31:12.032801+0000","last_became_active":"2024-04-22T06:31:11.045061+0000","last_became_peered":"2024-04-22T06:31:11.045061+0000","last_unstale":"2024-04-22T06:31:12.032801+0000","last_undegraded":"2024-04-22T06:31:12.032801+0000","last_fullsized":"2024-04-22T06:31:12.032801+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:00:33.958921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559690+0000","last_change":"2024-04-22T06:31:07.026506+0000","last_active":"2024-04-22T06:31:12.559690+0000","last_peered":"2024-04-22T06:31:12.559690+0000","last_clean":"2024-04-22T06:31:12.559690+0000","last_became_active":"2024-04-22T06:31:07.026174+0000","last_became_peered":"2024-04-22T06:31:07.026174+0000","last_unstale":"2024-04-22T06:31:12.559690+0000","last_undegraded":"2024-04-22T06:31:12.559690+0000","last_fullsized":"2024-04-22T06:31:12.559690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:35:21.002170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560793+0000","last_change":"2024-04-22T06:31:05.997990+0000","last_active":"2024-04-22T06:31:12.560793+0000","last_peered":"2024-04-22T06:31:12.560793+0000","last_clean":"2024-04-22T06:31:12.560793+0000","last_became_active":"2024-04-22T06:31:05.997352+0000","last_became_peered":"2024-04-22T06:31:05.997352+0000","last_unstale":"2024-04-22T06:31:12.560793+0000","last_undegraded":"2024-04-22T06:31:12.560793+0000","last_fullsized":"2024-04-22T06:31:12.560793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:15:40.726807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559303+0000","last_change":"2024-04-22T06:31:11.053487+0000","last_active":"2024-04-22T06:31:12.559303+0000","last_peered":"2024-04-22T06:31:12.559303+0000","last_clean":"2024-04-22T06:31:12.559303+0000","last_became_active":"2024-04-22T06:31:11.053078+0000","last_became_peered":"2024-04-22T06:31:11.053078+0000","last_unstale":"2024-04-22T06:31:12.559303+0000","last_undegraded":"2024-04-22T06:31:12.559303+0000","last_fullsized":"2024-04-22T06:31:12.559303+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:23:16.130369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561518+0000","last_change":"2024-04-22T06:31:07.020420+0000","last_active":"2024-04-22T06:31:12.561518+0000","last_peered":"2024-04-22T06:31:12.561518+0000","last_clean":"2024-04-22T06:31:12.561518+0000","last_became_active":"2024-04-22T06:31:07.020110+0000","last_became_peered":"2024-04-22T06:31:07.020110+0000","last_unstale":"2024-04-22T06:31:12.561518+0000","last_undegraded":"2024-04-22T06:31:12.561518+0000","last_fullsized":"2024-04-22T06:31:12.561518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:44:23.997057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031731+0000","last_change":"2024-04-22T06:31:05.997357+0000","last_active":"2024-04-22T06:31:12.031731+0000","last_peered":"2024-04-22T06:31:12.031731+0000","last_clean":"2024-04-22T06:31:12.031731+0000","last_became_active":"2024-04-22T06:31:05.996913+0000","last_became_peered":"2024-04-22T06:31:05.996913+0000","last_unstale":"2024-04-22T06:31:12.031731+0000","last_undegraded":"2024-04-22T06:31:12.031731+0000","last_fullsized":"2024-04-22T06:31:12.031731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:50:51.880199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560195+0000","last_change":"2024-04-22T06:31:11.059735+0000","last_active":"2024-04-22T06:31:12.560195+0000","last_peered":"2024-04-22T06:31:12.560195+0000","last_clean":"2024-04-22T06:31:12.560195+0000","last_became_active":"2024-04-22T06:31:11.059325+0000","last_became_peered":"2024-04-22T06:31:11.059325+0000","last_unstale":"2024-04-22T06:31:12.560195+0000","last_undegraded":"2024-04-22T06:31:12.560195+0000","last_fullsized":"2024-04-22T06:31:12.560195+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:59:52.881783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.032101+0000","last_change":"2024-04-22T06:31:07.026576+0000","last_active":"2024-04-22T06:31:12.032101+0000","last_peered":"2024-04-22T06:31:12.032101+0000","last_clean":"2024-04-22T06:31:12.032101+0000","last_became_active":"2024-04-22T06:31:07.026057+0000","last_became_peered":"2024-04-22T06:31:07.026057+0000","last_unstale":"2024-04-22T06:31:12.032101+0000","last_undegraded":"2024-04-22T06:31:12.032101+0000","last_fullsized":"2024-04-22T06:31:12.032101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:49:32.525918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561588+0000","last_change":"2024-04-22T06:31:06.009043+0000","last_active":"2024-04-22T06:31:12.561588+0000","last_peered":"2024-04-22T06:31:12.561588+0000","last_clean":"2024-04-22T06:31:12.561588+0000","last_became_active":"2024-04-22T06:31:06.008752+0000","last_became_peered":"2024-04-22T06:31:06.008752+0000","last_unstale":"2024-04-22T06:31:12.561588+0000","last_undegraded":"2024-04-22T06:31:12.561588+0000","last_fullsized":"2024-04-22T06:31:12.561588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:52:34.706288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561059+0000","last_change":"2024-04-22T06:31:11.057688+0000","last_active":"2024-04-22T06:31:12.561059+0000","last_peered":"2024-04-22T06:31:12.561059+0000","last_clean":"2024-04-22T06:31:12.561059+0000","last_became_active":"2024-04-22T06:31:11.057354+0000","last_became_peered":"2024-04-22T06:31:11.057354+0000","last_unstale":"2024-04-22T06:31:12.561059+0000","last_undegraded":"2024-04-22T06:31:12.561059+0000","last_fullsized":"2024-04-22T06:31:12.561059+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:08:36.080352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.033017+0000","last_change":"2024-04-22T06:31:07.028506+0000","last_active":"2024-04-22T06:31:12.033017+0000","last_peered":"2024-04-22T06:31:12.033017+0000","last_clean":"2024-04-22T06:31:12.033017+0000","last_became_active":"2024-04-22T06:31:07.028183+0000","last_became_peered":"2024-04-22T06:31:07.028183+0000","last_unstale":"2024-04-22T06:31:12.033017+0000","last_undegraded":"2024-04-22T06:31:12.033017+0000","last_fullsized":"2024-04-22T06:31:12.033017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:38:45.965934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031683+0000","last_change":"2024-04-22T06:31:06.002052+0000","last_active":"2024-04-22T06:31:12.031683+0000","last_peered":"2024-04-22T06:31:12.031683+0000","last_clean":"2024-04-22T06:31:12.031683+0000","last_became_active":"2024-04-22T06:31:06.000645+0000","last_became_peered":"2024-04-22T06:31:06.000645+0000","last_unstale":"2024-04-22T06:31:12.031683+0000","last_undegraded":"2024-04-22T06:31:12.031683+0000","last_fullsized":"2024-04-22T06:31:12.031683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:55:38.982563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561693+0000","last_change":"2024-04-22T06:31:11.048333+0000","last_active":"2024-04-22T06:31:12.561693+0000","last_peered":"2024-04-22T06:31:12.561693+0000","last_clean":"2024-04-22T06:31:12.561693+0000","last_became_active":"2024-04-22T06:31:11.047913+0000","last_became_peered":"2024-04-22T06:31:11.047913+0000","last_unstale":"2024-04-22T06:31:12.561693+0000","last_undegraded":"2024-04-22T06:31:12.561693+0000","last_fullsized":"2024-04-22T06:31:12.561693+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:11:52.622436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561265+0000","last_change":"2024-04-22T06:31:07.017430+0000","last_active":"2024-04-22T06:31:12.561265+0000","last_peered":"2024-04-22T06:31:12.561265+0000","last_clean":"2024-04-22T06:31:12.561265+0000","last_became_active":"2024-04-22T06:31:07.017112+0000","last_became_peered":"2024-04-22T06:31:07.017112+0000","last_unstale":"2024-04-22T06:31:12.561265+0000","last_undegraded":"2024-04-22T06:31:12.561265+0000","last_fullsized":"2024-04-22T06:31:12.561265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:52:18.098301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558848+0000","last_change":"2024-04-22T06:31:06.002202+0000","last_active":"2024-04-22T06:31:12.558848+0000","last_peered":"2024-04-22T06:31:12.558848+0000","last_clean":"2024-04-22T06:31:12.558848+0000","last_became_active":"2024-04-22T06:31:06.001916+0000","last_became_peered":"2024-04-22T06:31:06.001916+0000","last_unstale":"2024-04-22T06:31:12.558848+0000","last_undegraded":"2024-04-22T06:31:12.558848+0000","last_fullsized":"2024-04-22T06:31:12.558848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:35:52.485268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560894+0000","last_change":"2024-04-22T06:31:11.054576+0000","last_active":"2024-04-22T06:31:12.560894+0000","last_peered":"2024-04-22T06:31:12.560894+0000","last_clean":"2024-04-22T06:31:12.560894+0000","last_became_active":"2024-04-22T06:31:11.054158+0000","last_became_peered":"2024-04-22T06:31:11.054158+0000","last_unstale":"2024-04-22T06:31:12.560894+0000","last_undegraded":"2024-04-22T06:31:12.560894+0000","last_fullsized":"2024-04-22T06:31:12.560894+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:48:50.809440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558837+0000","last_change":"2024-04-22T06:31:07.023064+0000","last_active":"2024-04-22T06:31:12.558837+0000","last_peered":"2024-04-22T06:31:12.558837+0000","last_clean":"2024-04-22T06:31:12.558837+0000","last_became_active":"2024-04-22T06:31:07.022753+0000","last_became_peered":"2024-04-22T06:31:07.022753+0000","last_unstale":"2024-04-22T06:31:12.558837+0000","last_undegraded":"2024-04-22T06:31:12.558837+0000","last_fullsized":"2024-04-22T06:31:12.558837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:25:38.039689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.029946+0000","last_change":"2024-04-22T06:31:05.998384+0000","last_active":"2024-04-22T06:31:12.029946+0000","last_peered":"2024-04-22T06:31:12.029946+0000","last_clean":"2024-04-22T06:31:12.029946+0000","last_became_active":"2024-04-22T06:31:05.997761+0000","last_became_peered":"2024-04-22T06:31:05.997761+0000","last_unstale":"2024-04-22T06:31:12.029946+0000","last_undegraded":"2024-04-22T06:31:12.029946+0000","last_fullsized":"2024-04-22T06:31:12.029946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:07:12.014505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561016+0000","last_change":"2024-04-22T06:31:11.060877+0000","last_active":"2024-04-22T06:31:12.561016+0000","last_peered":"2024-04-22T06:31:12.561016+0000","last_clean":"2024-04-22T06:31:12.561016+0000","last_became_active":"2024-04-22T06:31:11.060473+0000","last_became_peered":"2024-04-22T06:31:11.060473+0000","last_unstale":"2024-04-22T06:31:12.561016+0000","last_undegraded":"2024-04-22T06:31:12.561016+0000","last_fullsized":"2024-04-22T06:31:12.561016+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:47:18.456791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558817+0000","last_change":"2024-04-22T06:31:07.026139+0000","last_active":"2024-04-22T06:31:12.558817+0000","last_peered":"2024-04-22T06:31:12.558817+0000","last_clean":"2024-04-22T06:31:12.558817+0000","last_became_active":"2024-04-22T06:31:07.025826+0000","last_became_peered":"2024-04-22T06:31:07.025826+0000","last_unstale":"2024-04-22T06:31:12.558817+0000","last_undegraded":"2024-04-22T06:31:12.558817+0000","last_fullsized":"2024-04-22T06:31:12.558817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:58:48.200581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031641+0000","last_change":"2024-04-22T06:31:06.001815+0000","last_active":"2024-04-22T06:31:12.031641+0000","last_peered":"2024-04-22T06:31:12.031641+0000","last_clean":"2024-04-22T06:31:12.031641+0000","last_became_active":"2024-04-22T06:31:06.000629+0000","last_became_peered":"2024-04-22T06:31:06.000629+0000","last_unstale":"2024-04-22T06:31:12.031641+0000","last_undegraded":"2024-04-22T06:31:12.031641+0000","last_fullsized":"2024-04-22T06:31:12.031641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:49:31.100969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030036+0000","last_change":"2024-04-22T06:30:56.214982+0000","last_active":"2024-04-22T06:31:12.030036+0000","last_peered":"2024-04-22T06:31:12.030036+0000","last_clean":"2024-04-22T06:31:12.030036+0000","last_became_active":"2024-04-22T06:30:56.214024+0000","last_became_peered":"2024-04-22T06:30:56.214024+0000","last_unstale":"2024-04-22T06:31:12.030036+0000","last_undegraded":"2024-04-22T06:31:12.030036+0000","last_fullsized":"2024-04-22T06:31:12.030036+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:30:54.947156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:30:54.947156+0000","last_clean_scrub_stamp":"2024-04-22T06:30:54.947156+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:32:37.920238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558994+0000","last_change":"2024-04-22T06:31:11.040152+0000","last_active":"2024-04-22T06:31:12.558994+0000","last_peered":"2024-04-22T06:31:12.558994+0000","last_clean":"2024-04-22T06:31:12.558994+0000","last_became_active":"2024-04-22T06:31:11.039749+0000","last_became_peered":"2024-04-22T06:31:11.039749+0000","last_unstale":"2024-04-22T06:31:12.558994+0000","last_undegraded":"2024-04-22T06:31:12.558994+0000","last_fullsized":"2024-04-22T06:31:12.558994+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:31:23.103876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031283+0000","last_change":"2024-04-22T06:31:07.011062+0000","last_active":"2024-04-22T06:31:12.031283+0000","last_peered":"2024-04-22T06:31:12.031283+0000","last_clean":"2024-04-22T06:31:12.031283+0000","last_became_active":"2024-04-22T06:31:07.010755+0000","last_became_peered":"2024-04-22T06:31:07.010755+0000","last_unstale":"2024-04-22T06:31:12.031283+0000","last_undegraded":"2024-04-22T06:31:12.031283+0000","last_fullsized":"2024-04-22T06:31:12.031283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:57:01.373036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560069+0000","last_change":"2024-04-22T06:31:05.998083+0000","last_active":"2024-04-22T06:31:12.560069+0000","last_peered":"2024-04-22T06:31:12.560069+0000","last_clean":"2024-04-22T06:31:12.560069+0000","last_became_active":"2024-04-22T06:31:05.997777+0000","last_became_peered":"2024-04-22T06:31:05.997777+0000","last_unstale":"2024-04-22T06:31:12.560069+0000","last_undegraded":"2024-04-22T06:31:12.560069+0000","last_fullsized":"2024-04-22T06:31:12.560069+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:15:13.171198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558666+0000","last_change":"2024-04-22T06:31:11.049607+0000","last_active":"2024-04-22T06:31:12.558666+0000","last_peered":"2024-04-22T06:31:12.558666+0000","last_clean":"2024-04-22T06:31:12.558666+0000","last_became_active":"2024-04-22T06:31:11.049156+0000","last_became_peered":"2024-04-22T06:31:11.049156+0000","last_unstale":"2024-04-22T06:31:12.558666+0000","last_undegraded":"2024-04-22T06:31:12.558666+0000","last_fullsized":"2024-04-22T06:31:12.558666+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:15:48.599278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.032032+0000","last_change":"2024-04-22T06:31:07.026306+0000","last_active":"2024-04-22T06:31:12.032032+0000","last_peered":"2024-04-22T06:31:12.032032+0000","last_clean":"2024-04-22T06:31:12.032032+0000","last_became_active":"2024-04-22T06:31:07.025987+0000","last_became_peered":"2024-04-22T06:31:07.025987+0000","last_unstale":"2024-04-22T06:31:12.032032+0000","last_undegraded":"2024-04-22T06:31:12.032032+0000","last_fullsized":"2024-04-22T06:31:12.032032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:50:31.803180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560942+0000","last_change":"2024-04-22T06:31:06.003115+0000","last_active":"2024-04-22T06:31:12.560942+0000","last_peered":"2024-04-22T06:31:12.560942+0000","last_clean":"2024-04-22T06:31:12.560942+0000","last_became_active":"2024-04-22T06:31:06.001945+0000","last_became_peered":"2024-04-22T06:31:06.001945+0000","last_unstale":"2024-04-22T06:31:12.560942+0000","last_undegraded":"2024-04-22T06:31:12.560942+0000","last_fullsized":"2024-04-22T06:31:12.560942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:59:25.770504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561105+0000","last_change":"2024-04-22T06:31:11.058841+0000","last_active":"2024-04-22T06:31:12.561105+0000","last_peered":"2024-04-22T06:31:12.561105+0000","last_clean":"2024-04-22T06:31:12.561105+0000","last_became_active":"2024-04-22T06:31:11.058407+0000","last_became_peered":"2024-04-22T06:31:11.058407+0000","last_unstale":"2024-04-22T06:31:12.561105+0000","last_undegraded":"2024-04-22T06:31:12.561105+0000","last_fullsized":"2024-04-22T06:31:12.561105+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:30:20.432210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031032+0000","last_change":"2024-04-22T06:31:07.022314+0000","last_active":"2024-04-22T06:31:12.031032+0000","last_peered":"2024-04-22T06:31:12.031032+0000","last_clean":"2024-04-22T06:31:12.031032+0000","last_became_active":"2024-04-22T06:31:07.021497+0000","last_became_peered":"2024-04-22T06:31:07.021497+0000","last_unstale":"2024-04-22T06:31:12.031032+0000","last_undegraded":"2024-04-22T06:31:12.031032+0000","last_fullsized":"2024-04-22T06:31:12.031032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:16:25.960359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559348+0000","last_change":"2024-04-22T06:31:06.003805+0000","last_active":"2024-04-22T06:31:12.559348+0000","last_peered":"2024-04-22T06:31:12.559348+0000","last_clean":"2024-04-22T06:31:12.559348+0000","last_became_active":"2024-04-22T06:31:06.002735+0000","last_became_peered":"2024-04-22T06:31:06.002735+0000","last_unstale":"2024-04-22T06:31:12.559348+0000","last_undegraded":"2024-04-22T06:31:12.559348+0000","last_fullsized":"2024-04-22T06:31:12.559348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:41:16.155628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030380+0000","last_change":"2024-04-22T06:31:11.043003+0000","last_active":"2024-04-22T06:31:12.030380+0000","last_peered":"2024-04-22T06:31:12.030380+0000","last_clean":"2024-04-22T06:31:12.030380+0000","last_became_active":"2024-04-22T06:31:11.042599+0000","last_became_peered":"2024-04-22T06:31:11.042599+0000","last_unstale":"2024-04-22T06:31:12.030380+0000","last_undegraded":"2024-04-22T06:31:12.030380+0000","last_fullsized":"2024-04-22T06:31:12.030380+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:28:58.949214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030520+0000","last_change":"2024-04-22T06:31:07.021258+0000","last_active":"2024-04-22T06:31:12.030520+0000","last_peered":"2024-04-22T06:31:12.030520+0000","last_clean":"2024-04-22T06:31:12.030520+0000","last_became_active":"2024-04-22T06:31:07.020976+0000","last_became_peered":"2024-04-22T06:31:07.020976+0000","last_unstale":"2024-04-22T06:31:12.030520+0000","last_undegraded":"2024-04-22T06:31:12.030520+0000","last_fullsized":"2024-04-22T06:31:12.030520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:19:33.377626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"24'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.033565+0000","last_change":"2024-04-22T06:31:06.002751+0000","last_active":"2024-04-22T06:31:12.033565+0000","last_peered":"2024-04-22T06:31:12.033565+0000","last_clean":"2024-04-22T06:31:12.033565+0000","last_became_active":"2024-04-22T06:31:06.002489+0000","last_became_peered":"2024-04-22T06:31:06.002489+0000","last_unstale":"2024-04-22T06:31:12.033565+0000","last_undegraded":"2024-04-22T06:31:12.033565+0000","last_fullsized":"2024-04-22T06:31:12.033565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:20:05.937529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.032720+0000","last_change":"2024-04-22T06:31:11.048954+0000","last_active":"2024-04-22T06:31:12.032720+0000","last_peered":"2024-04-22T06:31:12.032720+0000","last_clean":"2024-04-22T06:31:12.032720+0000","last_became_active":"2024-04-22T06:31:11.048214+0000","last_became_peered":"2024-04-22T06:31:11.048214+0000","last_unstale":"2024-04-22T06:31:12.032720+0000","last_undegraded":"2024-04-22T06:31:12.032720+0000","last_fullsized":"2024-04-22T06:31:12.032720+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:13:21.903203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030103+0000","last_change":"2024-04-22T06:31:07.021258+0000","last_active":"2024-04-22T06:31:12.030103+0000","last_peered":"2024-04-22T06:31:12.030103+0000","last_clean":"2024-04-22T06:31:12.030103+0000","last_became_active":"2024-04-22T06:31:07.021036+0000","last_became_peered":"2024-04-22T06:31:07.021036+0000","last_unstale":"2024-04-22T06:31:12.030103+0000","last_undegraded":"2024-04-22T06:31:12.030103+0000","last_fullsized":"2024-04-22T06:31:12.030103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:51:39.990228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"24'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559515+0000","last_change":"2024-04-22T06:31:05.998270+0000","last_active":"2024-04-22T06:31:12.559515+0000","last_peered":"2024-04-22T06:31:12.559515+0000","last_clean":"2024-04-22T06:31:12.559515+0000","last_became_active":"2024-04-22T06:31:05.997925+0000","last_became_peered":"2024-04-22T06:31:05.997925+0000","last_unstale":"2024-04-22T06:31:12.559515+0000","last_undegraded":"2024-04-22T06:31:12.559515+0000","last_fullsized":"2024-04-22T06:31:12.559515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:49:44.167932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560845+0000","last_change":"2024-04-22T06:31:11.051106+0000","last_active":"2024-04-22T06:31:12.560845+0000","last_peered":"2024-04-22T06:31:12.560845+0000","last_clean":"2024-04-22T06:31:12.560845+0000","last_became_active":"2024-04-22T06:31:11.050721+0000","last_became_peered":"2024-04-22T06:31:11.050721+0000","last_unstale":"2024-04-22T06:31:12.560845+0000","last_undegraded":"2024-04-22T06:31:12.560845+0000","last_fullsized":"2024-04-22T06:31:12.560845+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:58:43.521850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558584+0000","last_change":"2024-04-22T06:31:07.020281+0000","last_active":"2024-04-22T06:31:12.558584+0000","last_peered":"2024-04-22T06:31:12.558584+0000","last_clean":"2024-04-22T06:31:12.558584+0000","last_became_active":"2024-04-22T06:31:07.020012+0000","last_became_peered":"2024-04-22T06:31:07.020012+0000","last_unstale":"2024-04-22T06:31:12.558584+0000","last_undegraded":"2024-04-22T06:31:12.558584+0000","last_fullsized":"2024-04-22T06:31:12.558584+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:20:40.301631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558743+0000","last_change":"2024-04-22T06:31:05.999918+0000","last_active":"2024-04-22T06:31:12.558743+0000","last_peered":"2024-04-22T06:31:12.558743+0000","last_clean":"2024-04-22T06:31:12.558743+0000","last_became_active":"2024-04-22T06:31:05.999594+0000","last_became_peered":"2024-04-22T06:31:05.999594+0000","last_unstale":"2024-04-22T06:31:12.558743+0000","last_undegraded":"2024-04-22T06:31:12.558743+0000","last_fullsized":"2024-04-22T06:31:12.558743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:15:21.376196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559535+0000","last_change":"2024-04-22T06:31:11.055940+0000","last_active":"2024-04-22T06:31:12.559535+0000","last_peered":"2024-04-22T06:31:12.559535+0000","last_clean":"2024-04-22T06:31:12.559535+0000","last_became_active":"2024-04-22T06:31:11.054873+0000","last_became_peered":"2024-04-22T06:31:11.054873+0000","last_unstale":"2024-04-22T06:31:12.559535+0000","last_undegraded":"2024-04-22T06:31:12.559535+0000","last_fullsized":"2024-04-22T06:31:12.559535+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:13:51.061199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030857+0000","last_change":"2024-04-22T06:31:07.021839+0000","last_active":"2024-04-22T06:31:12.030857+0000","last_peered":"2024-04-22T06:31:12.030857+0000","last_clean":"2024-04-22T06:31:12.030857+0000","last_became_active":"2024-04-22T06:31:07.021585+0000","last_became_peered":"2024-04-22T06:31:07.021585+0000","last_unstale":"2024-04-22T06:31:12.030857+0000","last_undegraded":"2024-04-22T06:31:12.030857+0000","last_fullsized":"2024-04-22T06:31:12.030857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:55:15.275802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031584+0000","last_change":"2024-04-22T06:31:06.004058+0000","last_active":"2024-04-22T06:31:12.031584+0000","last_peered":"2024-04-22T06:31:12.031584+0000","last_clean":"2024-04-22T06:31:12.031584+0000","last_became_active":"2024-04-22T06:31:06.003791+0000","last_became_peered":"2024-04-22T06:31:06.003791+0000","last_unstale":"2024-04-22T06:31:12.031584+0000","last_undegraded":"2024-04-22T06:31:12.031584+0000","last_fullsized":"2024-04-22T06:31:12.031584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:03:08.913113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559058+0000","last_change":"2024-04-22T06:31:11.054245+0000","last_active":"2024-04-22T06:31:12.559058+0000","last_peered":"2024-04-22T06:31:12.559058+0000","last_clean":"2024-04-22T06:31:12.559058+0000","last_became_active":"2024-04-22T06:31:11.053551+0000","last_became_peered":"2024-04-22T06:31:11.053551+0000","last_unstale":"2024-04-22T06:31:12.559058+0000","last_undegraded":"2024-04-22T06:31:12.559058+0000","last_fullsized":"2024-04-22T06:31:12.559058+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:34:06.312700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559478+0000","last_change":"2024-04-22T06:31:07.023273+0000","last_active":"2024-04-22T06:31:12.559478+0000","last_peered":"2024-04-22T06:31:12.559478+0000","last_clean":"2024-04-22T06:31:12.559478+0000","last_became_active":"2024-04-22T06:31:07.022978+0000","last_became_peered":"2024-04-22T06:31:07.022978+0000","last_unstale":"2024-04-22T06:31:12.559478+0000","last_undegraded":"2024-04-22T06:31:12.559478+0000","last_fullsized":"2024-04-22T06:31:12.559478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:10:50.192298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"24'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561165+0000","last_change":"2024-04-22T06:31:06.005510+0000","last_active":"2024-04-22T06:31:12.561165+0000","last_peered":"2024-04-22T06:31:12.561165+0000","last_clean":"2024-04-22T06:31:12.561165+0000","last_became_active":"2024-04-22T06:31:06.005047+0000","last_became_peered":"2024-04-22T06:31:06.005047+0000","last_unstale":"2024-04-22T06:31:12.561165+0000","last_undegraded":"2024-04-22T06:31:12.561165+0000","last_fullsized":"2024-04-22T06:31:12.561165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:25:51.227631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.032100+0000","last_change":"2024-04-22T06:31:11.046209+0000","last_active":"2024-04-22T06:31:12.032100+0000","last_peered":"2024-04-22T06:31:12.032100+0000","last_clean":"2024-04-22T06:31:12.032100+0000","last_became_active":"2024-04-22T06:31:11.045339+0000","last_became_peered":"2024-04-22T06:31:11.045339+0000","last_unstale":"2024-04-22T06:31:12.032100+0000","last_undegraded":"2024-04-22T06:31:12.032100+0000","last_fullsized":"2024-04-22T06:31:12.032100+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:39:16.659010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031172+0000","last_change":"2024-04-22T06:31:07.025901+0000","last_active":"2024-04-22T06:31:12.031172+0000","last_peered":"2024-04-22T06:31:12.031172+0000","last_clean":"2024-04-22T06:31:12.031172+0000","last_became_active":"2024-04-22T06:31:07.025590+0000","last_became_peered":"2024-04-22T06:31:07.025590+0000","last_unstale":"2024-04-22T06:31:12.031172+0000","last_undegraded":"2024-04-22T06:31:12.031172+0000","last_fullsized":"2024-04-22T06:31:12.031172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:00:26.184054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:19.272923+0000","last_change":"2024-04-22T06:31:06.004609+0000","last_active":"2024-04-22T06:31:19.272923+0000","last_peered":"2024-04-22T06:31:19.272923+0000","last_clean":"2024-04-22T06:31:19.272923+0000","last_became_active":"2024-04-22T06:31:06.004319+0000","last_became_peered":"2024-04-22T06:31:06.004319+0000","last_unstale":"2024-04-22T06:31:19.272923+0000","last_undegraded":"2024-04-22T06:31:19.272923+0000","last_fullsized":"2024-04-22T06:31:19.272923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:32:47.198906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561266+0000","last_change":"2024-04-22T06:31:11.054706+0000","last_active":"2024-04-22T06:31:12.561266+0000","last_peered":"2024-04-22T06:31:12.561266+0000","last_clean":"2024-04-22T06:31:12.561266+0000","last_became_active":"2024-04-22T06:31:11.054307+0000","last_became_peered":"2024-04-22T06:31:11.054307+0000","last_unstale":"2024-04-22T06:31:12.561266+0000","last_undegraded":"2024-04-22T06:31:12.561266+0000","last_fullsized":"2024-04-22T06:31:12.561266+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:05:02.465114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.033403+0000","last_change":"2024-04-22T06:31:07.020739+0000","last_active":"2024-04-22T06:31:12.033403+0000","last_peered":"2024-04-22T06:31:12.033403+0000","last_clean":"2024-04-22T06:31:12.033403+0000","last_became_active":"2024-04-22T06:31:07.020070+0000","last_became_peered":"2024-04-22T06:31:07.020070+0000","last_unstale":"2024-04-22T06:31:12.033403+0000","last_undegraded":"2024-04-22T06:31:12.033403+0000","last_fullsized":"2024-04-22T06:31:12.033403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:50:22.102753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"24'3","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560805+0000","last_change":"2024-04-22T06:31:06.003733+0000","last_active":"2024-04-22T06:31:12.560805+0000","last_peered":"2024-04-22T06:31:12.560805+0000","last_clean":"2024-04-22T06:31:12.560805+0000","last_became_active":"2024-04-22T06:31:06.003465+0000","last_became_peered":"2024-04-22T06:31:06.003465+0000","last_unstale":"2024-04-22T06:31:12.560805+0000","last_undegraded":"2024-04-22T06:31:12.560805+0000","last_fullsized":"2024-04-22T06:31:12.560805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:19:36.023311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.032875+0000","last_change":"2024-04-22T06:31:11.058041+0000","last_active":"2024-04-22T06:31:12.032875+0000","last_peered":"2024-04-22T06:31:12.032875+0000","last_clean":"2024-04-22T06:31:12.032875+0000","last_became_active":"2024-04-22T06:31:11.057243+0000","last_became_peered":"2024-04-22T06:31:11.057243+0000","last_unstale":"2024-04-22T06:31:12.032875+0000","last_undegraded":"2024-04-22T06:31:12.032875+0000","last_fullsized":"2024-04-22T06:31:12.032875+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:49:06.781140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560767+0000","last_change":"2024-04-22T06:31:07.015369+0000","last_active":"2024-04-22T06:31:12.560767+0000","last_peered":"2024-04-22T06:31:12.560767+0000","last_clean":"2024-04-22T06:31:12.560767+0000","last_became_active":"2024-04-22T06:31:07.015088+0000","last_became_peered":"2024-04-22T06:31:07.015088+0000","last_unstale":"2024-04-22T06:31:12.560767+0000","last_undegraded":"2024-04-22T06:31:12.560767+0000","last_fullsized":"2024-04-22T06:31:12.560767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:10:09.449507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030766+0000","last_change":"2024-04-22T06:31:05.999636+0000","last_active":"2024-04-22T06:31:12.030766+0000","last_peered":"2024-04-22T06:31:12.030766+0000","last_clean":"2024-04-22T06:31:12.030766+0000","last_became_active":"2024-04-22T06:31:05.999329+0000","last_became_peered":"2024-04-22T06:31:05.999329+0000","last_unstale":"2024-04-22T06:31:12.030766+0000","last_undegraded":"2024-04-22T06:31:12.030766+0000","last_fullsized":"2024-04-22T06:31:12.030766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:09:06.670827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559595+0000","last_change":"2024-04-22T06:31:11.053530+0000","last_active":"2024-04-22T06:31:12.559595+0000","last_peered":"2024-04-22T06:31:12.559595+0000","last_clean":"2024-04-22T06:31:12.559595+0000","last_became_active":"2024-04-22T06:31:11.053100+0000","last_became_peered":"2024-04-22T06:31:11.053100+0000","last_unstale":"2024-04-22T06:31:12.559595+0000","last_undegraded":"2024-04-22T06:31:12.559595+0000","last_fullsized":"2024-04-22T06:31:12.559595+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:57:16.506840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030657+0000","last_change":"2024-04-22T06:31:07.015838+0000","last_active":"2024-04-22T06:31:12.030657+0000","last_peered":"2024-04-22T06:31:12.030657+0000","last_clean":"2024-04-22T06:31:12.030657+0000","last_became_active":"2024-04-22T06:31:07.015545+0000","last_became_peered":"2024-04-22T06:31:07.015545+0000","last_unstale":"2024-04-22T06:31:12.030657+0000","last_undegraded":"2024-04-22T06:31:12.030657+0000","last_fullsized":"2024-04-22T06:31:12.030657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:43:54.251061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559092+0000","last_change":"2024-04-22T06:31:06.005442+0000","last_active":"2024-04-22T06:31:12.559092+0000","last_peered":"2024-04-22T06:31:12.559092+0000","last_clean":"2024-04-22T06:31:12.559092+0000","last_became_active":"2024-04-22T06:31:06.005156+0000","last_became_peered":"2024-04-22T06:31:06.005156+0000","last_unstale":"2024-04-22T06:31:12.559092+0000","last_undegraded":"2024-04-22T06:31:12.559092+0000","last_fullsized":"2024-04-22T06:31:12.559092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:52:32.138519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560401+0000","last_change":"2024-04-22T06:31:11.046885+0000","last_active":"2024-04-22T06:31:12.560401+0000","last_peered":"2024-04-22T06:31:12.560401+0000","last_clean":"2024-04-22T06:31:12.560401+0000","last_became_active":"2024-04-22T06:31:11.046444+0000","last_became_peered":"2024-04-22T06:31:11.046444+0000","last_unstale":"2024-04-22T06:31:12.560401+0000","last_undegraded":"2024-04-22T06:31:12.560401+0000","last_fullsized":"2024-04-22T06:31:12.560401+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:17:54.523416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560687+0000","last_change":"2024-04-22T06:31:11.061784+0000","last_active":"2024-04-22T06:31:12.560687+0000","last_peered":"2024-04-22T06:31:12.560687+0000","last_clean":"2024-04-22T06:31:12.560687+0000","last_became_active":"2024-04-22T06:31:11.061382+0000","last_became_peered":"2024-04-22T06:31:11.061382+0000","last_unstale":"2024-04-22T06:31:12.560687+0000","last_undegraded":"2024-04-22T06:31:12.560687+0000","last_fullsized":"2024-04-22T06:31:12.560687+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:12:53.829206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560353+0000","last_change":"2024-04-22T06:31:07.023542+0000","last_active":"2024-04-22T06:31:12.560353+0000","last_peered":"2024-04-22T06:31:12.560353+0000","last_clean":"2024-04-22T06:31:12.560353+0000","last_became_active":"2024-04-22T06:31:07.023240+0000","last_became_peered":"2024-04-22T06:31:07.023240+0000","last_unstale":"2024-04-22T06:31:12.560353+0000","last_undegraded":"2024-04-22T06:31:12.560353+0000","last_fullsized":"2024-04-22T06:31:12.560353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:15:34.727195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030673+0000","last_change":"2024-04-22T06:31:06.003080+0000","last_active":"2024-04-22T06:31:12.030673+0000","last_peered":"2024-04-22T06:31:12.030673+0000","last_clean":"2024-04-22T06:31:12.030673+0000","last_became_active":"2024-04-22T06:31:06.002725+0000","last_became_peered":"2024-04-22T06:31:06.002725+0000","last_unstale":"2024-04-22T06:31:12.030673+0000","last_undegraded":"2024-04-22T06:31:12.030673+0000","last_fullsized":"2024-04-22T06:31:12.030673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:46:38.249137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559100+0000","last_change":"2024-04-22T06:31:11.047782+0000","last_active":"2024-04-22T06:31:12.559100+0000","last_peered":"2024-04-22T06:31:12.559100+0000","last_clean":"2024-04-22T06:31:12.559100+0000","last_became_active":"2024-04-22T06:31:11.047407+0000","last_became_peered":"2024-04-22T06:31:11.047407+0000","last_unstale":"2024-04-22T06:31:12.559100+0000","last_undegraded":"2024-04-22T06:31:12.559100+0000","last_fullsized":"2024-04-22T06:31:12.559100+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:48:05.416994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559643+0000","last_change":"2024-04-22T06:31:07.025799+0000","last_active":"2024-04-22T06:31:12.559643+0000","last_peered":"2024-04-22T06:31:12.559643+0000","last_clean":"2024-04-22T06:31:12.559643+0000","last_became_active":"2024-04-22T06:31:07.025420+0000","last_became_peered":"2024-04-22T06:31:07.025420+0000","last_unstale":"2024-04-22T06:31:12.559643+0000","last_undegraded":"2024-04-22T06:31:12.559643+0000","last_fullsized":"2024-04-22T06:31:12.559643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:39:40.107621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560547+0000","last_change":"2024-04-22T06:31:06.005805+0000","last_active":"2024-04-22T06:31:12.560547+0000","last_peered":"2024-04-22T06:31:12.560547+0000","last_clean":"2024-04-22T06:31:12.560547+0000","last_became_active":"2024-04-22T06:31:06.005479+0000","last_became_peered":"2024-04-22T06:31:06.005479+0000","last_unstale":"2024-04-22T06:31:12.560547+0000","last_undegraded":"2024-04-22T06:31:12.560547+0000","last_fullsized":"2024-04-22T06:31:12.560547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:59:37.140612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"24'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558740+0000","last_change":"2024-04-22T06:31:05.998354+0000","last_active":"2024-04-22T06:31:12.558740+0000","last_peered":"2024-04-22T06:31:12.558740+0000","last_clean":"2024-04-22T06:31:12.558740+0000","last_became_active":"2024-04-22T06:31:05.998071+0000","last_became_peered":"2024-04-22T06:31:05.998071+0000","last_unstale":"2024-04-22T06:31:12.558740+0000","last_undegraded":"2024-04-22T06:31:12.558740+0000","last_fullsized":"2024-04-22T06:31:12.558740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:12:13.343435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561078+0000","last_change":"2024-04-22T06:31:07.024109+0000","last_active":"2024-04-22T06:31:12.561078+0000","last_peered":"2024-04-22T06:31:12.561078+0000","last_clean":"2024-04-22T06:31:12.561078+0000","last_became_active":"2024-04-22T06:31:07.023346+0000","last_became_peered":"2024-04-22T06:31:07.023346+0000","last_unstale":"2024-04-22T06:31:12.561078+0000","last_undegraded":"2024-04-22T06:31:12.561078+0000","last_fullsized":"2024-04-22T06:31:12.561078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:02:42.662312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559888+0000","last_change":"2024-04-22T06:31:11.047132+0000","last_active":"2024-04-22T06:31:12.559888+0000","last_peered":"2024-04-22T06:31:12.559888+0000","last_clean":"2024-04-22T06:31:12.559888+0000","last_became_active":"2024-04-22T06:31:11.046671+0000","last_became_peered":"2024-04-22T06:31:11.046671+0000","last_unstale":"2024-04-22T06:31:12.559888+0000","last_undegraded":"2024-04-22T06:31:12.559888+0000","last_fullsized":"2024-04-22T06:31:12.559888+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:41:57.706735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"24'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561258+0000","last_change":"2024-04-22T06:31:06.004971+0000","last_active":"2024-04-22T06:31:12.561258+0000","last_peered":"2024-04-22T06:31:12.561258+0000","last_clean":"2024-04-22T06:31:12.561258+0000","last_became_active":"2024-04-22T06:31:06.004473+0000","last_became_peered":"2024-04-22T06:31:06.004473+0000","last_unstale":"2024-04-22T06:31:12.561258+0000","last_undegraded":"2024-04-22T06:31:12.561258+0000","last_fullsized":"2024-04-22T06:31:12.561258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:47:11.735311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.032950+0000","last_change":"2024-04-22T06:31:07.018314+0000","last_active":"2024-04-22T06:31:12.032950+0000","last_peered":"2024-04-22T06:31:12.032950+0000","last_clean":"2024-04-22T06:31:12.032950+0000","last_became_active":"2024-04-22T06:31:07.018033+0000","last_became_peered":"2024-04-22T06:31:07.018033+0000","last_unstale":"2024-04-22T06:31:12.032950+0000","last_undegraded":"2024-04-22T06:31:12.032950+0000","last_fullsized":"2024-04-22T06:31:12.032950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:06:06.099494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559988+0000","last_change":"2024-04-22T06:31:11.055874+0000","last_active":"2024-04-22T06:31:12.559988+0000","last_peered":"2024-04-22T06:31:12.559988+0000","last_clean":"2024-04-22T06:31:12.559988+0000","last_became_active":"2024-04-22T06:31:11.055371+0000","last_became_peered":"2024-04-22T06:31:11.055371+0000","last_unstale":"2024-04-22T06:31:12.559988+0000","last_undegraded":"2024-04-22T06:31:12.559988+0000","last_fullsized":"2024-04-22T06:31:12.559988+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:06:03.909337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558461+0000","last_change":"2024-04-22T06:31:07.024780+0000","last_active":"2024-04-22T06:31:12.558461+0000","last_peered":"2024-04-22T06:31:12.558461+0000","last_clean":"2024-04-22T06:31:12.558461+0000","last_became_active":"2024-04-22T06:31:07.024501+0000","last_became_peered":"2024-04-22T06:31:07.024501+0000","last_unstale":"2024-04-22T06:31:12.558461+0000","last_undegraded":"2024-04-22T06:31:12.558461+0000","last_fullsized":"2024-04-22T06:31:12.558461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:12:45.981232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558747+0000","last_change":"2024-04-22T06:31:07.014546+0000","last_active":"2024-04-22T06:31:12.558747+0000","last_peered":"2024-04-22T06:31:12.558747+0000","last_clean":"2024-04-22T06:31:12.558747+0000","last_became_active":"2024-04-22T06:31:07.014196+0000","last_became_peered":"2024-04-22T06:31:07.014196+0000","last_unstale":"2024-04-22T06:31:12.558747+0000","last_undegraded":"2024-04-22T06:31:12.558747+0000","last_fullsized":"2024-04-22T06:31:12.558747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:57:33.136875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031490+0000","last_change":"2024-04-22T06:31:07.018563+0000","last_active":"2024-04-22T06:31:12.031490+0000","last_peered":"2024-04-22T06:31:12.031490+0000","last_clean":"2024-04-22T06:31:12.031490+0000","last_became_active":"2024-04-22T06:31:07.018289+0000","last_became_peered":"2024-04-22T06:31:07.018289+0000","last_unstale":"2024-04-22T06:31:12.031490+0000","last_undegraded":"2024-04-22T06:31:12.031490+0000","last_fullsized":"2024-04-22T06:31:12.031490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:30:11.892544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560983+0000","last_change":"2024-04-22T06:31:07.022901+0000","last_active":"2024-04-22T06:31:12.560983+0000","last_peered":"2024-04-22T06:31:12.560983+0000","last_clean":"2024-04-22T06:31:12.560983+0000","last_became_active":"2024-04-22T06:31:07.022420+0000","last_became_peered":"2024-04-22T06:31:07.022420+0000","last_unstale":"2024-04-22T06:31:12.560983+0000","last_undegraded":"2024-04-22T06:31:12.560983+0000","last_fullsized":"2024-04-22T06:31:12.560983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:24:18.929896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559253+0000","last_change":"2024-04-22T06:31:07.022875+0000","last_active":"2024-04-22T06:31:12.559253+0000","last_peered":"2024-04-22T06:31:12.559253+0000","last_clean":"2024-04-22T06:31:12.559253+0000","last_became_active":"2024-04-22T06:31:07.022545+0000","last_became_peered":"2024-04-22T06:31:07.022545+0000","last_unstale":"2024-04-22T06:31:12.559253+0000","last_undegraded":"2024-04-22T06:31:12.559253+0000","last_fullsized":"2024-04-22T06:31:12.559253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:02:30.464508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560354+0000","last_change":"2024-04-22T06:31:07.026367+0000","last_active":"2024-04-22T06:31:12.560354+0000","last_peered":"2024-04-22T06:31:12.560354+0000","last_clean":"2024-04-22T06:31:12.560354+0000","last_became_active":"2024-04-22T06:31:07.026085+0000","last_became_peered":"2024-04-22T06:31:07.026085+0000","last_unstale":"2024-04-22T06:31:12.560354+0000","last_undegraded":"2024-04-22T06:31:12.560354+0000","last_fullsized":"2024-04-22T06:31:12.560354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:02:55.368591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559243+0000","last_change":"2024-04-22T06:31:07.022989+0000","last_active":"2024-04-22T06:31:12.559243+0000","last_peered":"2024-04-22T06:31:12.559243+0000","last_clean":"2024-04-22T06:31:12.559243+0000","last_became_active":"2024-04-22T06:31:07.022688+0000","last_became_peered":"2024-04-22T06:31:07.022688+0000","last_unstale":"2024-04-22T06:31:12.559243+0000","last_undegraded":"2024-04-22T06:31:12.559243+0000","last_fullsized":"2024-04-22T06:31:12.559243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:24:10.910731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559590+0000","last_change":"2024-04-22T06:31:07.026315+0000","last_active":"2024-04-22T06:31:12.559590+0000","last_peered":"2024-04-22T06:31:12.559590+0000","last_clean":"2024-04-22T06:31:12.559590+0000","last_became_active":"2024-04-22T06:31:07.025951+0000","last_became_peered":"2024-04-22T06:31:07.025951+0000","last_unstale":"2024-04-22T06:31:12.559590+0000","last_undegraded":"2024-04-22T06:31:12.559590+0000","last_fullsized":"2024-04-22T06:31:12.559590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:54:12.160058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030585+0000","last_change":"2024-04-22T06:31:07.022748+0000","last_active":"2024-04-22T06:31:12.030585+0000","last_peered":"2024-04-22T06:31:12.030585+0000","last_clean":"2024-04-22T06:31:12.030585+0000","last_became_active":"2024-04-22T06:31:07.022481+0000","last_became_peered":"2024-04-22T06:31:07.022481+0000","last_unstale":"2024-04-22T06:31:12.030585+0000","last_undegraded":"2024-04-22T06:31:12.030585+0000","last_fullsized":"2024-04-22T06:31:12.030585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:56:32.631637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560456+0000","last_change":"2024-04-22T06:31:07.021939+0000","last_active":"2024-04-22T06:31:12.560456+0000","last_peered":"2024-04-22T06:31:12.560456+0000","last_clean":"2024-04-22T06:31:12.560456+0000","last_became_active":"2024-04-22T06:31:07.021638+0000","last_became_peered":"2024-04-22T06:31:07.021638+0000","last_unstale":"2024-04-22T06:31:12.560456+0000","last_undegraded":"2024-04-22T06:31:12.560456+0000","last_fullsized":"2024-04-22T06:31:12.560456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:05:27.106092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031390+0000","last_change":"2024-04-22T06:31:07.015844+0000","last_active":"2024-04-22T06:31:12.031390+0000","last_peered":"2024-04-22T06:31:12.031390+0000","last_clean":"2024-04-22T06:31:12.031390+0000","last_became_active":"2024-04-22T06:31:07.015520+0000","last_became_peered":"2024-04-22T06:31:07.015520+0000","last_unstale":"2024-04-22T06:31:12.031390+0000","last_undegraded":"2024-04-22T06:31:12.031390+0000","last_fullsized":"2024-04-22T06:31:12.031390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:24:04.676892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560128+0000","last_change":"2024-04-22T06:31:07.023697+0000","last_active":"2024-04-22T06:31:12.560128+0000","last_peered":"2024-04-22T06:31:12.560128+0000","last_clean":"2024-04-22T06:31:12.560128+0000","last_became_active":"2024-04-22T06:31:07.023283+0000","last_became_peered":"2024-04-22T06:31:07.023283+0000","last_unstale":"2024-04-22T06:31:12.560128+0000","last_undegraded":"2024-04-22T06:31:12.560128+0000","last_fullsized":"2024-04-22T06:31:12.560128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:43:01.338104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558868+0000","last_change":"2024-04-22T06:31:07.023953+0000","last_active":"2024-04-22T06:31:12.558868+0000","last_peered":"2024-04-22T06:31:12.558868+0000","last_clean":"2024-04-22T06:31:12.558868+0000","last_became_active":"2024-04-22T06:31:07.023704+0000","last_became_peered":"2024-04-22T06:31:07.023704+0000","last_unstale":"2024-04-22T06:31:12.558868+0000","last_undegraded":"2024-04-22T06:31:12.558868+0000","last_fullsized":"2024-04-22T06:31:12.558868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:59:32.219492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031619+0000","last_change":"2024-04-22T06:31:07.027418+0000","last_active":"2024-04-22T06:31:12.031619+0000","last_peered":"2024-04-22T06:31:12.031619+0000","last_clean":"2024-04-22T06:31:12.031619+0000","last_became_active":"2024-04-22T06:31:07.027113+0000","last_became_peered":"2024-04-22T06:31:07.027113+0000","last_unstale":"2024-04-22T06:31:12.031619+0000","last_undegraded":"2024-04-22T06:31:12.031619+0000","last_fullsized":"2024-04-22T06:31:12.031619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:08:42.204674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559261+0000","last_change":"2024-04-22T06:31:07.015520+0000","last_active":"2024-04-22T06:31:12.559261+0000","last_peered":"2024-04-22T06:31:12.559261+0000","last_clean":"2024-04-22T06:31:12.559261+0000","last_became_active":"2024-04-22T06:31:07.015199+0000","last_became_peered":"2024-04-22T06:31:07.015199+0000","last_unstale":"2024-04-22T06:31:12.559261+0000","last_undegraded":"2024-04-22T06:31:12.559261+0000","last_fullsized":"2024-04-22T06:31:12.559261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:48:40.688931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031281+0000","last_change":"2024-04-22T06:31:07.016457+0000","last_active":"2024-04-22T06:31:12.031281+0000","last_peered":"2024-04-22T06:31:12.031281+0000","last_clean":"2024-04-22T06:31:12.031281+0000","last_became_active":"2024-04-22T06:31:07.016082+0000","last_became_peered":"2024-04-22T06:31:07.016082+0000","last_unstale":"2024-04-22T06:31:12.031281+0000","last_undegraded":"2024-04-22T06:31:12.031281+0000","last_fullsized":"2024-04-22T06:31:12.031281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:12:28.517065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030497+0000","last_change":"2024-04-22T06:31:07.019955+0000","last_active":"2024-04-22T06:31:12.030497+0000","last_peered":"2024-04-22T06:31:12.030497+0000","last_clean":"2024-04-22T06:31:12.030497+0000","last_became_active":"2024-04-22T06:31:07.019174+0000","last_became_peered":"2024-04-22T06:31:07.019174+0000","last_unstale":"2024-04-22T06:31:12.030497+0000","last_undegraded":"2024-04-22T06:31:12.030497+0000","last_fullsized":"2024-04-22T06:31:12.030497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:09:07.106161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560474+0000","last_change":"2024-04-22T06:31:07.019341+0000","last_active":"2024-04-22T06:31:12.560474+0000","last_peered":"2024-04-22T06:31:12.560474+0000","last_clean":"2024-04-22T06:31:12.560474+0000","last_became_active":"2024-04-22T06:31:07.019048+0000","last_became_peered":"2024-04-22T06:31:07.019048+0000","last_unstale":"2024-04-22T06:31:12.560474+0000","last_undegraded":"2024-04-22T06:31:12.560474+0000","last_fullsized":"2024-04-22T06:31:12.560474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:04:19.362686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559337+0000","last_change":"2024-04-22T06:31:07.019433+0000","last_active":"2024-04-22T06:31:12.559337+0000","last_peered":"2024-04-22T06:31:12.559337+0000","last_clean":"2024-04-22T06:31:12.559337+0000","last_became_active":"2024-04-22T06:31:07.019133+0000","last_became_peered":"2024-04-22T06:31:07.019133+0000","last_unstale":"2024-04-22T06:31:12.559337+0000","last_undegraded":"2024-04-22T06:31:12.559337+0000","last_fullsized":"2024-04-22T06:31:12.559337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:01:54.466486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561087+0000","last_change":"2024-04-22T06:31:07.023014+0000","last_active":"2024-04-22T06:31:12.561087+0000","last_peered":"2024-04-22T06:31:12.561087+0000","last_clean":"2024-04-22T06:31:12.561087+0000","last_became_active":"2024-04-22T06:31:07.022716+0000","last_became_peered":"2024-04-22T06:31:07.022716+0000","last_unstale":"2024-04-22T06:31:12.561087+0000","last_undegraded":"2024-04-22T06:31:12.561087+0000","last_fullsized":"2024-04-22T06:31:12.561087+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:11:47.309706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.033363+0000","last_change":"2024-04-22T06:31:07.020446+0000","last_active":"2024-04-22T06:31:12.033363+0000","last_peered":"2024-04-22T06:31:12.033363+0000","last_clean":"2024-04-22T06:31:12.033363+0000","last_became_active":"2024-04-22T06:31:07.018995+0000","last_became_peered":"2024-04-22T06:31:07.018995+0000","last_unstale":"2024-04-22T06:31:12.033363+0000","last_undegraded":"2024-04-22T06:31:12.033363+0000","last_fullsized":"2024-04-22T06:31:12.033363+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:56:45.031262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558848+0000","last_change":"2024-04-22T06:31:07.014927+0000","last_active":"2024-04-22T06:31:12.558848+0000","last_peered":"2024-04-22T06:31:12.558848+0000","last_clean":"2024-04-22T06:31:12.558848+0000","last_became_active":"2024-04-22T06:31:07.014652+0000","last_became_peered":"2024-04-22T06:31:07.014652+0000","last_unstale":"2024-04-22T06:31:12.558848+0000","last_undegraded":"2024-04-22T06:31:12.558848+0000","last_fullsized":"2024-04-22T06:31:12.558848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:51:33.506667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559161+0000","last_change":"2024-04-22T06:31:07.014209+0000","last_active":"2024-04-22T06:31:12.559161+0000","last_peered":"2024-04-22T06:31:12.559161+0000","last_clean":"2024-04-22T06:31:12.559161+0000","last_became_active":"2024-04-22T06:31:07.013946+0000","last_became_peered":"2024-04-22T06:31:07.013946+0000","last_unstale":"2024-04-22T06:31:12.559161+0000","last_undegraded":"2024-04-22T06:31:12.559161+0000","last_fullsized":"2024-04-22T06:31:12.559161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:40:31.669158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031004+0000","last_change":"2024-04-22T06:31:07.008764+0000","last_active":"2024-04-22T06:31:12.031004+0000","last_peered":"2024-04-22T06:31:12.031004+0000","last_clean":"2024-04-22T06:31:12.031004+0000","last_became_active":"2024-04-22T06:31:07.007810+0000","last_became_peered":"2024-04-22T06:31:07.007810+0000","last_unstale":"2024-04-22T06:31:12.031004+0000","last_undegraded":"2024-04-22T06:31:12.031004+0000","last_fullsized":"2024-04-22T06:31:12.031004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:49:54.894818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559408+0000","last_change":"2024-04-22T06:31:07.022141+0000","last_active":"2024-04-22T06:31:12.559408+0000","last_peered":"2024-04-22T06:31:12.559408+0000","last_clean":"2024-04-22T06:31:12.559408+0000","last_became_active":"2024-04-22T06:31:07.021845+0000","last_became_peered":"2024-04-22T06:31:07.021845+0000","last_unstale":"2024-04-22T06:31:12.559408+0000","last_undegraded":"2024-04-22T06:31:12.559408+0000","last_fullsized":"2024-04-22T06:31:12.559408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:26:28.598104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.029984+0000","last_change":"2024-04-22T06:31:07.012188+0000","last_active":"2024-04-22T06:31:12.029984+0000","last_peered":"2024-04-22T06:31:12.029984+0000","last_clean":"2024-04-22T06:31:12.029984+0000","last_became_active":"2024-04-22T06:31:07.011909+0000","last_became_peered":"2024-04-22T06:31:07.011909+0000","last_unstale":"2024-04-22T06:31:12.029984+0000","last_undegraded":"2024-04-22T06:31:12.029984+0000","last_fullsized":"2024-04-22T06:31:12.029984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:25:13.089928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031293+0000","last_change":"2024-04-22T06:31:07.020033+0000","last_active":"2024-04-22T06:31:12.031293+0000","last_peered":"2024-04-22T06:31:12.031293+0000","last_clean":"2024-04-22T06:31:12.031293+0000","last_became_active":"2024-04-22T06:31:07.019689+0000","last_became_peered":"2024-04-22T06:31:07.019689+0000","last_unstale":"2024-04-22T06:31:12.031293+0000","last_undegraded":"2024-04-22T06:31:12.031293+0000","last_fullsized":"2024-04-22T06:31:12.031293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:09:07.209203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030666+0000","last_change":"2024-04-22T06:31:07.027287+0000","last_active":"2024-04-22T06:31:12.030666+0000","last_peered":"2024-04-22T06:31:12.030666+0000","last_clean":"2024-04-22T06:31:12.030666+0000","last_became_active":"2024-04-22T06:31:07.027015+0000","last_became_peered":"2024-04-22T06:31:07.027015+0000","last_unstale":"2024-04-22T06:31:12.030666+0000","last_undegraded":"2024-04-22T06:31:12.030666+0000","last_fullsized":"2024-04-22T06:31:12.030666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:42:49.322413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031959+0000","last_change":"2024-04-22T06:31:07.026332+0000","last_active":"2024-04-22T06:31:12.031959+0000","last_peered":"2024-04-22T06:31:12.031959+0000","last_clean":"2024-04-22T06:31:12.031959+0000","last_became_active":"2024-04-22T06:31:07.026018+0000","last_became_peered":"2024-04-22T06:31:07.026018+0000","last_unstale":"2024-04-22T06:31:12.031959+0000","last_undegraded":"2024-04-22T06:31:12.031959+0000","last_fullsized":"2024-04-22T06:31:12.031959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:27:08.560056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560402+0000","last_change":"2024-04-22T06:31:07.020497+0000","last_active":"2024-04-22T06:31:12.560402+0000","last_peered":"2024-04-22T06:31:12.560402+0000","last_clean":"2024-04-22T06:31:12.560402+0000","last_became_active":"2024-04-22T06:31:07.020194+0000","last_became_peered":"2024-04-22T06:31:07.020194+0000","last_unstale":"2024-04-22T06:31:12.560402+0000","last_undegraded":"2024-04-22T06:31:12.560402+0000","last_fullsized":"2024-04-22T06:31:12.560402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:56:59.765295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560664+0000","last_change":"2024-04-22T06:31:07.018396+0000","last_active":"2024-04-22T06:31:12.560664+0000","last_peered":"2024-04-22T06:31:12.560664+0000","last_clean":"2024-04-22T06:31:12.560664+0000","last_became_active":"2024-04-22T06:31:07.018073+0000","last_became_peered":"2024-04-22T06:31:07.018073+0000","last_unstale":"2024-04-22T06:31:12.560664+0000","last_undegraded":"2024-04-22T06:31:12.560664+0000","last_fullsized":"2024-04-22T06:31:12.560664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:37:06.501499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030917+0000","last_change":"2024-04-22T06:31:07.014439+0000","last_active":"2024-04-22T06:31:12.030917+0000","last_peered":"2024-04-22T06:31:12.030917+0000","last_clean":"2024-04-22T06:31:12.030917+0000","last_became_active":"2024-04-22T06:31:07.014124+0000","last_became_peered":"2024-04-22T06:31:07.014124+0000","last_unstale":"2024-04-22T06:31:12.030917+0000","last_undegraded":"2024-04-22T06:31:12.030917+0000","last_fullsized":"2024-04-22T06:31:12.030917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:58:46.075068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":864256,"data_stored":695877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":270336,"data_stored":105017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":278528,"data_stored":105053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":104927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":274432,"data_stored":105087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":872448,"data_stored":695933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T06:31:35.095 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T06:31:35.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.232+0000 7f923e414640 1 Processor -- start 2024-04-22T06:31:35.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.232+0000 7f923e414640 1 -- start start 2024-04-22T06:31:35.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.233+0000 7f923e414640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f9238155240 0x7f923815f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:35.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.233+0000 7f923e414640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92380cc480 0x7f92380cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:35.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.233+0000 7f923e414640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92380cce30 con 0x7f92380cb320 2024-04-22T06:31:35.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.233+0000 7f923e414640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f923815f610 con 0x7f9238155240 2024-04-22T06:31:35.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.233+0000 7f923e414640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f923815c060 con 0x7f92380cc480 2024-04-22T06:31:35.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.233+0000 7f9237fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f9238155240 0x7f923815f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:35.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.233+0000 7f9237fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f9238155240 0x7f923815f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53416/0 (socket says 172.21.15.44:53416) 2024-04-22T06:31:35.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.233+0000 7f9237fff640 1 -- 172.21.15.44:0/4153644887 learned_addr learned my addr 172.21.15.44:0/4153644887 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:35.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.233+0000 7f923c9b1640 1 --2- 172.21.15.44:0/4153644887 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92380cc480 0x7f92380cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:35.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.233+0000 7f9237fff640 1 -- 172.21.15.44:0/4153644887 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92380cc480 msgr2=0x7f92380cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:35.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.233+0000 7f9237fff640 1 --2- 172.21.15.44:0/4153644887 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92380cc480 0x7f92380cc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:35.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.233+0000 7f9237fff640 1 -- 172.21.15.44:0/4153644887 >> v1:172.21.15.70:6789/0 conn(0x7f92380cb320 legacy=0x7f92380d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:35.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.233+0000 7f9237fff640 1 -- 172.21.15.44:0/4153644887 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f923815b200 con 0x7f9238155240 2024-04-22T06:31:35.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.234+0000 7f9237fff640 1 --2- 172.21.15.44:0/4153644887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9238155240 0x7f923815f0d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f92280098a0 tx=0x7f922802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc97bfe6ab1c762 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:35.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.234+0000 7f9236ffd640 1 -- 172.21.15.44:0/4153644887 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9228035020 con 0x7f9238155240 2024-04-22T06:31:35.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.234+0000 7f9236ffd640 1 -- 172.21.15.44:0/4153644887 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9228031640 con 0x7f9238155240 2024-04-22T06:31:35.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.234+0000 7f9236ffd640 1 -- 172.21.15.44:0/4153644887 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922802d040 con 0x7f9238155240 2024-04-22T06:31:35.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.235+0000 7f923e414640 1 -- 172.21.15.44:0/4153644887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9238155240 msgr2=0x7f923815f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:35.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.235+0000 7f923e414640 1 --2- 172.21.15.44:0/4153644887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9238155240 0x7f923815f0d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f92280098a0 tx=0x7f922802b8f0 comp rx=0 tx=0).stop 2024-04-22T06:31:35.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.235+0000 7f923e414640 1 -- 172.21.15.44:0/4153644887 shutdown_connections 2024-04-22T06:31:35.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.235+0000 7f923e414640 1 --2- 172.21.15.44:0/4153644887 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92380cc480 0x7f92380cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:35.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.235+0000 7f923e414640 1 --2- 172.21.15.44:0/4153644887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9238155240 0x7f923815f0d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:35.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.235+0000 7f923e414640 1 -- 172.21.15.44:0/4153644887 >> 172.21.15.44:0/4153644887 conn(0x7f92380750d0 msgr2=0x7f92380754e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:35.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.235+0000 7f923e414640 1 -- 172.21.15.44:0/4153644887 shutdown_connections 2024-04-22T06:31:35.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.235+0000 7f923e414640 1 -- 172.21.15.44:0/4153644887 wait complete. 2024-04-22T06:31:35.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.236+0000 7f923e414640 1 Processor -- start 2024-04-22T06:31:35.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.236+0000 7f923e414640 1 -- start start 2024-04-22T06:31:35.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.237+0000 7f923e414640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92380cb320 0x7f92380ca470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:35.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.237+0000 7f923e414640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92380cc480 0x7f92380c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:35.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.237+0000 7f923e414640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9238155240 0x7f92381f23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:35.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.237+0000 7f923e414640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f923815baa0 con 0x7f92380cc480 2024-04-22T06:31:35.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.237+0000 7f923e414640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f923815b7c0 con 0x7f92380cb320 2024-04-22T06:31:35.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.237+0000 7f923e414640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f923815bd80 con 0x7f9238155240 2024-04-22T06:31:35.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.237+0000 7f92377fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92380cc480 0x7f92380c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:35.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.237+0000 7f92377fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92380cc480 0x7f92380c7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53420/0 (socket says 172.21.15.44:53420) 2024-04-22T06:31:35.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.237+0000 7f92377fe640 1 -- 172.21.15.44:0/1388698721 learned_addr learned my addr 172.21.15.44:0/1388698721 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:35.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.237+0000 7f9237fff640 1 --2- 172.21.15.44:0/1388698721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92380cb320 0x7f92380ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:35.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.237+0000 7f923c9b1640 1 --2- 172.21.15.44:0/1388698721 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9238155240 0x7f92381f23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:35.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.237+0000 7f92377fe640 1 -- 172.21.15.44:0/1388698721 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9238155240 msgr2=0x7f92381f23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:35.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.237+0000 7f92377fe640 1 --2- 172.21.15.44:0/1388698721 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9238155240 0x7f92381f23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:35.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.237+0000 7f92377fe640 1 -- 172.21.15.44:0/1388698721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92380cb320 msgr2=0x7f92380ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:35.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.237+0000 7f92377fe640 1 --2- 172.21.15.44:0/1388698721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92380cb320 0x7f92380ca470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:35.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.237+0000 7f92377fe640 1 -- 172.21.15.44:0/1388698721 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92380c8b30 con 0x7f92380cc480 2024-04-22T06:31:35.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.237+0000 7f92377fe640 1 --2- 172.21.15.44:0/1388698721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92380cc480 0x7f92380c7fa0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f922c00b3f0 tx=0x7f922c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:35.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.238+0000 7f9234ff9640 1 -- 172.21.15.44:0/1388698721 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922c015020 con 0x7f92380cc480 2024-04-22T06:31:35.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.238+0000 7f9234ff9640 1 -- 172.21.15.44:0/1388698721 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f922c011830 con 0x7f92380cc480 2024-04-22T06:31:35.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.238+0000 7f9234ff9640 1 -- 172.21.15.44:0/1388698721 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922c011c20 con 0x7f92380cc480 2024-04-22T06:31:35.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.238+0000 7f923e414640 1 -- 172.21.15.44:0/1388698721 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92380c8e10 con 0x7f92380cc480 2024-04-22T06:31:35.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.238+0000 7f923e414640 1 -- 172.21.15.44:0/1388698721 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92380c93d0 con 0x7f92380cc480 2024-04-22T06:31:35.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.239+0000 7f923e414640 1 -- 172.21.15.44:0/1388698721 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9238068cc0 con 0x7f92380cc480 2024-04-22T06:31:35.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.239+0000 7f9234ff9640 1 -- 172.21.15.44:0/1388698721 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f922c018020 con 0x7f92380cc480 2024-04-22T06:31:35.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.240+0000 7f9234ff9640 1 --2- 172.21.15.44:0/1388698721 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f922406c8f0 0x7f922406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:35.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.240+0000 7f9234ff9640 1 -- 172.21.15.44:0/1388698721 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f922c08bad0 con 0x7f92380cc480 2024-04-22T06:31:35.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.240+0000 7f9237fff640 1 --2- 172.21.15.44:0/1388698721 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f922406c8f0 0x7f922406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:35.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.241+0000 7f9237fff640 1 --2- 172.21.15.44:0/1388698721 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f922406c8f0 0x7f922406edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9228036000 tx=0x7f922803c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:35.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.244+0000 7f9234ff9640 1 -- 172.21.15.44:0/1388698721 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f922c01f030 con 0x7f92380cc480 2024-04-22T06:31:35.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.426+0000 7f923e414640 1 -- 172.21.15.44:0/1388698721 --> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f923815aa80 con 0x7f922406c8f0 2024-04-22T06:31:35.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.434+0000 7f9234ff9640 1 -- 172.21.15.44:0/1388698721 <== mgr.4111 v2:172.21.15.70:6832/2570466131 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f923815aa80 con 0x7f922406c8f0 2024-04-22T06:31:35.437 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:31:35.439 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-04-22T06:31:35.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.437+0000 7f923e414640 1 -- 172.21.15.44:0/1388698721 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f922406c8f0 msgr2=0x7f922406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:35.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.437+0000 7f923e414640 1 --2- 172.21.15.44:0/1388698721 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f922406c8f0 0x7f922406edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9228036000 tx=0x7f922803c040 comp rx=0 tx=0).stop 2024-04-22T06:31:35.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.437+0000 7f923e414640 1 -- 172.21.15.44:0/1388698721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92380cc480 msgr2=0x7f92380c7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:35.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.437+0000 7f923e414640 1 --2- 172.21.15.44:0/1388698721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92380cc480 0x7f92380c7fa0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f922c00b3f0 tx=0x7f922c00b8b0 comp rx=0 tx=0).stop 2024-04-22T06:31:35.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.437+0000 7f923e414640 1 -- 172.21.15.44:0/1388698721 shutdown_connections 2024-04-22T06:31:35.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.437+0000 7f923e414640 1 --2- 172.21.15.44:0/1388698721 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f922406c8f0 0x7f922406edb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:35.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.437+0000 7f923e414640 1 --2- 172.21.15.44:0/1388698721 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9238155240 0x7f92381f23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:35.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.437+0000 7f923e414640 1 --2- 172.21.15.44:0/1388698721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92380cc480 0x7f92380c7fa0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:35.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.437+0000 7f923e414640 1 --2- 172.21.15.44:0/1388698721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92380cb320 0x7f92380ca470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:35.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.437+0000 7f923e414640 1 -- 172.21.15.44:0/1388698721 >> 172.21.15.44:0/1388698721 conn(0x7f92380750d0 msgr2=0x7f92380c61e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:35.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.437+0000 7f923e414640 1 -- 172.21.15.44:0/1388698721 shutdown_connections 2024-04-22T06:31:35.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.437+0000 7f923e414640 1 -- 172.21.15.44:0/1388698721 wait complete. 2024-04-22T06:31:35.448 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-04-22T06:31:34.551837+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236148,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738572,"statfs":{"total":773094113280,"available":772852297728,"internally_reserved":0,"allocated":3383296,"data_stored":2029432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002999"},"pg_stats":[{"pgid":"2.1f","version":"24'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560936+0000","last_change":"2024-04-22T06:31:06.002169+0000","last_active":"2024-04-22T06:31:12.560936+0000","last_peered":"2024-04-22T06:31:12.560936+0000","last_clean":"2024-04-22T06:31:12.560936+0000","last_became_active":"2024-04-22T06:31:06.001899+0000","last_became_peered":"2024-04-22T06:31:06.001899+0000","last_unstale":"2024-04-22T06:31:12.560936+0000","last_undegraded":"2024-04-22T06:31:12.560936+0000","last_fullsized":"2024-04-22T06:31:12.560936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:11:48.056404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559298+0000","last_change":"2024-04-22T06:31:07.025982+0000","last_active":"2024-04-22T06:31:12.559298+0000","last_peered":"2024-04-22T06:31:12.559298+0000","last_clean":"2024-04-22T06:31:12.559298+0000","last_became_active":"2024-04-22T06:31:07.025650+0000","last_became_peered":"2024-04-22T06:31:07.025650+0000","last_unstale":"2024-04-22T06:31:12.559298+0000","last_undegraded":"2024-04-22T06:31:12.559298+0000","last_fullsized":"2024-04-22T06:31:12.559298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:58:58.029898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560389+0000","last_change":"2024-04-22T06:31:11.048083+0000","last_active":"2024-04-22T06:31:12.560389+0000","last_peered":"2024-04-22T06:31:12.560389+0000","last_clean":"2024-04-22T06:31:12.560389+0000","last_became_active":"2024-04-22T06:31:11.047682+0000","last_became_peered":"2024-04-22T06:31:11.047682+0000","last_unstale":"2024-04-22T06:31:12.560389+0000","last_undegraded":"2024-04-22T06:31:12.560389+0000","last_fullsized":"2024-04-22T06:31:12.560389+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:34:30.158935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"24'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559543+0000","last_change":"2024-04-22T06:31:06.007484+0000","last_active":"2024-04-22T06:31:12.559543+0000","last_peered":"2024-04-22T06:31:12.559543+0000","last_clean":"2024-04-22T06:31:12.559543+0000","last_became_active":"2024-04-22T06:31:06.007204+0000","last_became_peered":"2024-04-22T06:31:06.007204+0000","last_unstale":"2024-04-22T06:31:12.559543+0000","last_undegraded":"2024-04-22T06:31:12.559543+0000","last_fullsized":"2024-04-22T06:31:12.559543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:39:18.905813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560596+0000","last_change":"2024-04-22T06:31:07.014658+0000","last_active":"2024-04-22T06:31:12.560596+0000","last_peered":"2024-04-22T06:31:12.560596+0000","last_clean":"2024-04-22T06:31:12.560596+0000","last_became_active":"2024-04-22T06:31:07.014283+0000","last_became_peered":"2024-04-22T06:31:07.014283+0000","last_unstale":"2024-04-22T06:31:12.560596+0000","last_undegraded":"2024-04-22T06:31:12.560596+0000","last_fullsized":"2024-04-22T06:31:12.560596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:43:01.159439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.032692+0000","last_change":"2024-04-22T06:31:11.045114+0000","last_active":"2024-04-22T06:31:12.032692+0000","last_peered":"2024-04-22T06:31:12.032692+0000","last_clean":"2024-04-22T06:31:12.032692+0000","last_became_active":"2024-04-22T06:31:11.044728+0000","last_became_peered":"2024-04-22T06:31:11.044728+0000","last_unstale":"2024-04-22T06:31:12.032692+0000","last_undegraded":"2024-04-22T06:31:12.032692+0000","last_fullsized":"2024-04-22T06:31:12.032692+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:48:19.215184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"24'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031494+0000","last_change":"2024-04-22T06:31:05.995468+0000","last_active":"2024-04-22T06:31:12.031494+0000","last_peered":"2024-04-22T06:31:12.031494+0000","last_clean":"2024-04-22T06:31:12.031494+0000","last_became_active":"2024-04-22T06:31:05.995159+0000","last_became_peered":"2024-04-22T06:31:05.995159+0000","last_unstale":"2024-04-22T06:31:12.031494+0000","last_undegraded":"2024-04-22T06:31:12.031494+0000","last_fullsized":"2024-04-22T06:31:12.031494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:01:12.223718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031966+0000","last_change":"2024-04-22T06:31:07.020024+0000","last_active":"2024-04-22T06:31:12.031966+0000","last_peered":"2024-04-22T06:31:12.031966+0000","last_clean":"2024-04-22T06:31:12.031966+0000","last_became_active":"2024-04-22T06:31:07.019490+0000","last_became_peered":"2024-04-22T06:31:07.019490+0000","last_unstale":"2024-04-22T06:31:12.031966+0000","last_undegraded":"2024-04-22T06:31:12.031966+0000","last_fullsized":"2024-04-22T06:31:12.031966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:26:41.811978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.033365+0000","last_change":"2024-04-22T06:31:11.057556+0000","last_active":"2024-04-22T06:31:12.033365+0000","last_peered":"2024-04-22T06:31:12.033365+0000","last_clean":"2024-04-22T06:31:12.033365+0000","last_became_active":"2024-04-22T06:31:11.057174+0000","last_became_peered":"2024-04-22T06:31:11.057174+0000","last_unstale":"2024-04-22T06:31:12.033365+0000","last_undegraded":"2024-04-22T06:31:12.033365+0000","last_fullsized":"2024-04-22T06:31:12.033365+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:41:19.526548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030542+0000","last_change":"2024-04-22T06:31:05.996281+0000","last_active":"2024-04-22T06:31:12.030542+0000","last_peered":"2024-04-22T06:31:12.030542+0000","last_clean":"2024-04-22T06:31:12.030542+0000","last_became_active":"2024-04-22T06:31:05.995951+0000","last_became_peered":"2024-04-22T06:31:05.995951+0000","last_unstale":"2024-04-22T06:31:12.030542+0000","last_undegraded":"2024-04-22T06:31:12.030542+0000","last_fullsized":"2024-04-22T06:31:12.030542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:01:43.732947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.032029+0000","last_change":"2024-04-22T06:31:07.017755+0000","last_active":"2024-04-22T06:31:12.032029+0000","last_peered":"2024-04-22T06:31:12.032029+0000","last_clean":"2024-04-22T06:31:12.032029+0000","last_became_active":"2024-04-22T06:31:07.017420+0000","last_became_peered":"2024-04-22T06:31:07.017420+0000","last_unstale":"2024-04-22T06:31:12.032029+0000","last_undegraded":"2024-04-22T06:31:12.032029+0000","last_fullsized":"2024-04-22T06:31:12.032029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:30:57.913020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.032802+0000","last_change":"2024-04-22T06:31:11.049497+0000","last_active":"2024-04-22T06:31:12.032802+0000","last_peered":"2024-04-22T06:31:12.032802+0000","last_clean":"2024-04-22T06:31:12.032802+0000","last_became_active":"2024-04-22T06:31:11.049079+0000","last_became_peered":"2024-04-22T06:31:11.049079+0000","last_unstale":"2024-04-22T06:31:12.032802+0000","last_undegraded":"2024-04-22T06:31:12.032802+0000","last_fullsized":"2024-04-22T06:31:12.032802+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:29:02.299691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558561+0000","last_change":"2024-04-22T06:31:07.023043+0000","last_active":"2024-04-22T06:31:12.558561+0000","last_peered":"2024-04-22T06:31:12.558561+0000","last_clean":"2024-04-22T06:31:12.558561+0000","last_became_active":"2024-04-22T06:31:07.022747+0000","last_became_peered":"2024-04-22T06:31:07.022747+0000","last_unstale":"2024-04-22T06:31:12.558561+0000","last_undegraded":"2024-04-22T06:31:12.558561+0000","last_fullsized":"2024-04-22T06:31:12.558561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:52:33.626072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"24'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030958+0000","last_change":"2024-04-22T06:31:05.997936+0000","last_active":"2024-04-22T06:31:12.030958+0000","last_peered":"2024-04-22T06:31:12.030958+0000","last_clean":"2024-04-22T06:31:12.030958+0000","last_became_active":"2024-04-22T06:31:05.997618+0000","last_became_peered":"2024-04-22T06:31:05.997618+0000","last_unstale":"2024-04-22T06:31:12.030958+0000","last_undegraded":"2024-04-22T06:31:12.030958+0000","last_fullsized":"2024-04-22T06:31:12.030958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:43:00.396691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.033550+0000","last_change":"2024-04-22T06:31:11.047725+0000","last_active":"2024-04-22T06:31:12.033550+0000","last_peered":"2024-04-22T06:31:12.033550+0000","last_clean":"2024-04-22T06:31:12.033550+0000","last_became_active":"2024-04-22T06:31:11.047278+0000","last_became_peered":"2024-04-22T06:31:11.047278+0000","last_unstale":"2024-04-22T06:31:12.033550+0000","last_undegraded":"2024-04-22T06:31:12.033550+0000","last_fullsized":"2024-04-22T06:31:12.033550+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:32:56.201059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561171+0000","last_change":"2024-04-22T06:31:07.021250+0000","last_active":"2024-04-22T06:31:12.561171+0000","last_peered":"2024-04-22T06:31:12.561171+0000","last_clean":"2024-04-22T06:31:12.561171+0000","last_became_active":"2024-04-22T06:31:07.020876+0000","last_became_peered":"2024-04-22T06:31:07.020876+0000","last_unstale":"2024-04-22T06:31:12.561171+0000","last_undegraded":"2024-04-22T06:31:12.561171+0000","last_fullsized":"2024-04-22T06:31:12.561171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:21:31.210819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561332+0000","last_change":"2024-04-22T06:31:06.012775+0000","last_active":"2024-04-22T06:31:12.561332+0000","last_peered":"2024-04-22T06:31:12.561332+0000","last_clean":"2024-04-22T06:31:12.561332+0000","last_became_active":"2024-04-22T06:31:06.012474+0000","last_became_peered":"2024-04-22T06:31:06.012474+0000","last_unstale":"2024-04-22T06:31:12.561332+0000","last_undegraded":"2024-04-22T06:31:12.561332+0000","last_fullsized":"2024-04-22T06:31:12.561332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:09:48.197011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.033219+0000","last_change":"2024-04-22T06:31:11.057641+0000","last_active":"2024-04-22T06:31:12.033219+0000","last_peered":"2024-04-22T06:31:12.033219+0000","last_clean":"2024-04-22T06:31:12.033219+0000","last_became_active":"2024-04-22T06:31:11.057221+0000","last_became_peered":"2024-04-22T06:31:11.057221+0000","last_unstale":"2024-04-22T06:31:12.033219+0000","last_undegraded":"2024-04-22T06:31:12.033219+0000","last_fullsized":"2024-04-22T06:31:12.033219+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:14:57.713113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559730+0000","last_change":"2024-04-22T06:31:07.020822+0000","last_active":"2024-04-22T06:31:12.559730+0000","last_peered":"2024-04-22T06:31:12.559730+0000","last_clean":"2024-04-22T06:31:12.559730+0000","last_became_active":"2024-04-22T06:31:07.020484+0000","last_became_peered":"2024-04-22T06:31:07.020484+0000","last_unstale":"2024-04-22T06:31:12.559730+0000","last_undegraded":"2024-04-22T06:31:12.559730+0000","last_fullsized":"2024-04-22T06:31:12.559730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:18:26.522131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560581+0000","last_change":"2024-04-22T06:31:06.000000+0000","last_active":"2024-04-22T06:31:12.560581+0000","last_peered":"2024-04-22T06:31:12.560581+0000","last_clean":"2024-04-22T06:31:12.560581+0000","last_became_active":"2024-04-22T06:31:05.999663+0000","last_became_peered":"2024-04-22T06:31:05.999663+0000","last_unstale":"2024-04-22T06:31:12.560581+0000","last_undegraded":"2024-04-22T06:31:12.560581+0000","last_fullsized":"2024-04-22T06:31:12.560581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:42.805084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560901+0000","last_change":"2024-04-22T06:31:11.057991+0000","last_active":"2024-04-22T06:31:12.560901+0000","last_peered":"2024-04-22T06:31:12.560901+0000","last_clean":"2024-04-22T06:31:12.560901+0000","last_became_active":"2024-04-22T06:31:11.057595+0000","last_became_peered":"2024-04-22T06:31:11.057595+0000","last_unstale":"2024-04-22T06:31:12.560901+0000","last_undegraded":"2024-04-22T06:31:12.560901+0000","last_fullsized":"2024-04-22T06:31:12.560901+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:17:06.366663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.033006+0000","last_change":"2024-04-22T06:31:07.017303+0000","last_active":"2024-04-22T06:31:12.033006+0000","last_peered":"2024-04-22T06:31:12.033006+0000","last_clean":"2024-04-22T06:31:12.033006+0000","last_became_active":"2024-04-22T06:31:07.016990+0000","last_became_peered":"2024-04-22T06:31:07.016990+0000","last_unstale":"2024-04-22T06:31:12.033006+0000","last_undegraded":"2024-04-22T06:31:12.033006+0000","last_fullsized":"2024-04-22T06:31:12.033006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:51:38.275472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.029843+0000","last_change":"2024-04-22T06:31:05.997363+0000","last_active":"2024-04-22T06:31:12.029843+0000","last_peered":"2024-04-22T06:31:12.029843+0000","last_clean":"2024-04-22T06:31:12.029843+0000","last_became_active":"2024-04-22T06:31:05.996982+0000","last_became_peered":"2024-04-22T06:31:05.996982+0000","last_unstale":"2024-04-22T06:31:12.029843+0000","last_undegraded":"2024-04-22T06:31:12.029843+0000","last_fullsized":"2024-04-22T06:31:12.029843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:23:50.052130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.033517+0000","last_change":"2024-04-22T06:31:11.048507+0000","last_active":"2024-04-22T06:31:12.033517+0000","last_peered":"2024-04-22T06:31:12.033517+0000","last_clean":"2024-04-22T06:31:12.033517+0000","last_became_active":"2024-04-22T06:31:11.047722+0000","last_became_peered":"2024-04-22T06:31:11.047722+0000","last_unstale":"2024-04-22T06:31:12.033517+0000","last_undegraded":"2024-04-22T06:31:12.033517+0000","last_fullsized":"2024-04-22T06:31:12.033517+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:04:48.732029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561139+0000","last_change":"2024-04-22T06:31:07.021095+0000","last_active":"2024-04-22T06:31:12.561139+0000","last_peered":"2024-04-22T06:31:12.561139+0000","last_clean":"2024-04-22T06:31:12.561139+0000","last_became_active":"2024-04-22T06:31:07.020814+0000","last_became_peered":"2024-04-22T06:31:07.020814+0000","last_unstale":"2024-04-22T06:31:12.561139+0000","last_undegraded":"2024-04-22T06:31:12.561139+0000","last_fullsized":"2024-04-22T06:31:12.561139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:27:08.869392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"24'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559162+0000","last_change":"2024-04-22T06:31:05.997508+0000","last_active":"2024-04-22T06:31:12.559162+0000","last_peered":"2024-04-22T06:31:12.559162+0000","last_clean":"2024-04-22T06:31:12.559162+0000","last_became_active":"2024-04-22T06:31:05.997233+0000","last_became_peered":"2024-04-22T06:31:05.997233+0000","last_unstale":"2024-04-22T06:31:12.559162+0000","last_undegraded":"2024-04-22T06:31:12.559162+0000","last_fullsized":"2024-04-22T06:31:12.559162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:40:56.909549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.032801+0000","last_change":"2024-04-22T06:31:11.045424+0000","last_active":"2024-04-22T06:31:12.032801+0000","last_peered":"2024-04-22T06:31:12.032801+0000","last_clean":"2024-04-22T06:31:12.032801+0000","last_became_active":"2024-04-22T06:31:11.045061+0000","last_became_peered":"2024-04-22T06:31:11.045061+0000","last_unstale":"2024-04-22T06:31:12.032801+0000","last_undegraded":"2024-04-22T06:31:12.032801+0000","last_fullsized":"2024-04-22T06:31:12.032801+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:00:33.958921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559690+0000","last_change":"2024-04-22T06:31:07.026506+0000","last_active":"2024-04-22T06:31:12.559690+0000","last_peered":"2024-04-22T06:31:12.559690+0000","last_clean":"2024-04-22T06:31:12.559690+0000","last_became_active":"2024-04-22T06:31:07.026174+0000","last_became_peered":"2024-04-22T06:31:07.026174+0000","last_unstale":"2024-04-22T06:31:12.559690+0000","last_undegraded":"2024-04-22T06:31:12.559690+0000","last_fullsized":"2024-04-22T06:31:12.559690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:35:21.002170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560793+0000","last_change":"2024-04-22T06:31:05.997990+0000","last_active":"2024-04-22T06:31:12.560793+0000","last_peered":"2024-04-22T06:31:12.560793+0000","last_clean":"2024-04-22T06:31:12.560793+0000","last_became_active":"2024-04-22T06:31:05.997352+0000","last_became_peered":"2024-04-22T06:31:05.997352+0000","last_unstale":"2024-04-22T06:31:12.560793+0000","last_undegraded":"2024-04-22T06:31:12.560793+0000","last_fullsized":"2024-04-22T06:31:12.560793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:15:40.726807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559303+0000","last_change":"2024-04-22T06:31:11.053487+0000","last_active":"2024-04-22T06:31:12.559303+0000","last_peered":"2024-04-22T06:31:12.559303+0000","last_clean":"2024-04-22T06:31:12.559303+0000","last_became_active":"2024-04-22T06:31:11.053078+0000","last_became_peered":"2024-04-22T06:31:11.053078+0000","last_unstale":"2024-04-22T06:31:12.559303+0000","last_undegraded":"2024-04-22T06:31:12.559303+0000","last_fullsized":"2024-04-22T06:31:12.559303+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:23:16.130369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561518+0000","last_change":"2024-04-22T06:31:07.020420+0000","last_active":"2024-04-22T06:31:12.561518+0000","last_peered":"2024-04-22T06:31:12.561518+0000","last_clean":"2024-04-22T06:31:12.561518+0000","last_became_active":"2024-04-22T06:31:07.020110+0000","last_became_peered":"2024-04-22T06:31:07.020110+0000","last_unstale":"2024-04-22T06:31:12.561518+0000","last_undegraded":"2024-04-22T06:31:12.561518+0000","last_fullsized":"2024-04-22T06:31:12.561518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:44:23.997057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031731+0000","last_change":"2024-04-22T06:31:05.997357+0000","last_active":"2024-04-22T06:31:12.031731+0000","last_peered":"2024-04-22T06:31:12.031731+0000","last_clean":"2024-04-22T06:31:12.031731+0000","last_became_active":"2024-04-22T06:31:05.996913+0000","last_became_peered":"2024-04-22T06:31:05.996913+0000","last_unstale":"2024-04-22T06:31:12.031731+0000","last_undegraded":"2024-04-22T06:31:12.031731+0000","last_fullsized":"2024-04-22T06:31:12.031731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:50:51.880199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560195+0000","last_change":"2024-04-22T06:31:11.059735+0000","last_active":"2024-04-22T06:31:12.560195+0000","last_peered":"2024-04-22T06:31:12.560195+0000","last_clean":"2024-04-22T06:31:12.560195+0000","last_became_active":"2024-04-22T06:31:11.059325+0000","last_became_peered":"2024-04-22T06:31:11.059325+0000","last_unstale":"2024-04-22T06:31:12.560195+0000","last_undegraded":"2024-04-22T06:31:12.560195+0000","last_fullsized":"2024-04-22T06:31:12.560195+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:59:52.881783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.032101+0000","last_change":"2024-04-22T06:31:07.026576+0000","last_active":"2024-04-22T06:31:12.032101+0000","last_peered":"2024-04-22T06:31:12.032101+0000","last_clean":"2024-04-22T06:31:12.032101+0000","last_became_active":"2024-04-22T06:31:07.026057+0000","last_became_peered":"2024-04-22T06:31:07.026057+0000","last_unstale":"2024-04-22T06:31:12.032101+0000","last_undegraded":"2024-04-22T06:31:12.032101+0000","last_fullsized":"2024-04-22T06:31:12.032101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:49:32.525918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561588+0000","last_change":"2024-04-22T06:31:06.009043+0000","last_active":"2024-04-22T06:31:12.561588+0000","last_peered":"2024-04-22T06:31:12.561588+0000","last_clean":"2024-04-22T06:31:12.561588+0000","last_became_active":"2024-04-22T06:31:06.008752+0000","last_became_peered":"2024-04-22T06:31:06.008752+0000","last_unstale":"2024-04-22T06:31:12.561588+0000","last_undegraded":"2024-04-22T06:31:12.561588+0000","last_fullsized":"2024-04-22T06:31:12.561588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:52:34.706288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561059+0000","last_change":"2024-04-22T06:31:11.057688+0000","last_active":"2024-04-22T06:31:12.561059+0000","last_peered":"2024-04-22T06:31:12.561059+0000","last_clean":"2024-04-22T06:31:12.561059+0000","last_became_active":"2024-04-22T06:31:11.057354+0000","last_became_peered":"2024-04-22T06:31:11.057354+0000","last_unstale":"2024-04-22T06:31:12.561059+0000","last_undegraded":"2024-04-22T06:31:12.561059+0000","last_fullsized":"2024-04-22T06:31:12.561059+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:08:36.080352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.033017+0000","last_change":"2024-04-22T06:31:07.028506+0000","last_active":"2024-04-22T06:31:12.033017+0000","last_peered":"2024-04-22T06:31:12.033017+0000","last_clean":"2024-04-22T06:31:12.033017+0000","last_became_active":"2024-04-22T06:31:07.028183+0000","last_became_peered":"2024-04-22T06:31:07.028183+0000","last_unstale":"2024-04-22T06:31:12.033017+0000","last_undegraded":"2024-04-22T06:31:12.033017+0000","last_fullsized":"2024-04-22T06:31:12.033017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:38:45.965934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031683+0000","last_change":"2024-04-22T06:31:06.002052+0000","last_active":"2024-04-22T06:31:12.031683+0000","last_peered":"2024-04-22T06:31:12.031683+0000","last_clean":"2024-04-22T06:31:12.031683+0000","last_became_active":"2024-04-22T06:31:06.000645+0000","last_became_peered":"2024-04-22T06:31:06.000645+0000","last_unstale":"2024-04-22T06:31:12.031683+0000","last_undegraded":"2024-04-22T06:31:12.031683+0000","last_fullsized":"2024-04-22T06:31:12.031683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:55:38.982563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561693+0000","last_change":"2024-04-22T06:31:11.048333+0000","last_active":"2024-04-22T06:31:12.561693+0000","last_peered":"2024-04-22T06:31:12.561693+0000","last_clean":"2024-04-22T06:31:12.561693+0000","last_became_active":"2024-04-22T06:31:11.047913+0000","last_became_peered":"2024-04-22T06:31:11.047913+0000","last_unstale":"2024-04-22T06:31:12.561693+0000","last_undegraded":"2024-04-22T06:31:12.561693+0000","last_fullsized":"2024-04-22T06:31:12.561693+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:11:52.622436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561265+0000","last_change":"2024-04-22T06:31:07.017430+0000","last_active":"2024-04-22T06:31:12.561265+0000","last_peered":"2024-04-22T06:31:12.561265+0000","last_clean":"2024-04-22T06:31:12.561265+0000","last_became_active":"2024-04-22T06:31:07.017112+0000","last_became_peered":"2024-04-22T06:31:07.017112+0000","last_unstale":"2024-04-22T06:31:12.561265+0000","last_undegraded":"2024-04-22T06:31:12.561265+0000","last_fullsized":"2024-04-22T06:31:12.561265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:52:18.098301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558848+0000","last_change":"2024-04-22T06:31:06.002202+0000","last_active":"2024-04-22T06:31:12.558848+0000","last_peered":"2024-04-22T06:31:12.558848+0000","last_clean":"2024-04-22T06:31:12.558848+0000","last_became_active":"2024-04-22T06:31:06.001916+0000","last_became_peered":"2024-04-22T06:31:06.001916+0000","last_unstale":"2024-04-22T06:31:12.558848+0000","last_undegraded":"2024-04-22T06:31:12.558848+0000","last_fullsized":"2024-04-22T06:31:12.558848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:35:52.485268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560894+0000","last_change":"2024-04-22T06:31:11.054576+0000","last_active":"2024-04-22T06:31:12.560894+0000","last_peered":"2024-04-22T06:31:12.560894+0000","last_clean":"2024-04-22T06:31:12.560894+0000","last_became_active":"2024-04-22T06:31:11.054158+0000","last_became_peered":"2024-04-22T06:31:11.054158+0000","last_unstale":"2024-04-22T06:31:12.560894+0000","last_undegraded":"2024-04-22T06:31:12.560894+0000","last_fullsized":"2024-04-22T06:31:12.560894+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:48:50.809440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558837+0000","last_change":"2024-04-22T06:31:07.023064+0000","last_active":"2024-04-22T06:31:12.558837+0000","last_peered":"2024-04-22T06:31:12.558837+0000","last_clean":"2024-04-22T06:31:12.558837+0000","last_became_active":"2024-04-22T06:31:07.022753+0000","last_became_peered":"2024-04-22T06:31:07.022753+0000","last_unstale":"2024-04-22T06:31:12.558837+0000","last_undegraded":"2024-04-22T06:31:12.558837+0000","last_fullsized":"2024-04-22T06:31:12.558837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:25:38.039689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.029946+0000","last_change":"2024-04-22T06:31:05.998384+0000","last_active":"2024-04-22T06:31:12.029946+0000","last_peered":"2024-04-22T06:31:12.029946+0000","last_clean":"2024-04-22T06:31:12.029946+0000","last_became_active":"2024-04-22T06:31:05.997761+0000","last_became_peered":"2024-04-22T06:31:05.997761+0000","last_unstale":"2024-04-22T06:31:12.029946+0000","last_undegraded":"2024-04-22T06:31:12.029946+0000","last_fullsized":"2024-04-22T06:31:12.029946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:07:12.014505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561016+0000","last_change":"2024-04-22T06:31:11.060877+0000","last_active":"2024-04-22T06:31:12.561016+0000","last_peered":"2024-04-22T06:31:12.561016+0000","last_clean":"2024-04-22T06:31:12.561016+0000","last_became_active":"2024-04-22T06:31:11.060473+0000","last_became_peered":"2024-04-22T06:31:11.060473+0000","last_unstale":"2024-04-22T06:31:12.561016+0000","last_undegraded":"2024-04-22T06:31:12.561016+0000","last_fullsized":"2024-04-22T06:31:12.561016+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:47:18.456791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558817+0000","last_change":"2024-04-22T06:31:07.026139+0000","last_active":"2024-04-22T06:31:12.558817+0000","last_peered":"2024-04-22T06:31:12.558817+0000","last_clean":"2024-04-22T06:31:12.558817+0000","last_became_active":"2024-04-22T06:31:07.025826+0000","last_became_peered":"2024-04-22T06:31:07.025826+0000","last_unstale":"2024-04-22T06:31:12.558817+0000","last_undegraded":"2024-04-22T06:31:12.558817+0000","last_fullsized":"2024-04-22T06:31:12.558817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:58:48.200581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031641+0000","last_change":"2024-04-22T06:31:06.001815+0000","last_active":"2024-04-22T06:31:12.031641+0000","last_peered":"2024-04-22T06:31:12.031641+0000","last_clean":"2024-04-22T06:31:12.031641+0000","last_became_active":"2024-04-22T06:31:06.000629+0000","last_became_peered":"2024-04-22T06:31:06.000629+0000","last_unstale":"2024-04-22T06:31:12.031641+0000","last_undegraded":"2024-04-22T06:31:12.031641+0000","last_fullsized":"2024-04-22T06:31:12.031641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:49:31.100969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030036+0000","last_change":"2024-04-22T06:30:56.214982+0000","last_active":"2024-04-22T06:31:12.030036+0000","last_peered":"2024-04-22T06:31:12.030036+0000","last_clean":"2024-04-22T06:31:12.030036+0000","last_became_active":"2024-04-22T06:30:56.214024+0000","last_became_peered":"2024-04-22T06:30:56.214024+0000","last_unstale":"2024-04-22T06:31:12.030036+0000","last_undegraded":"2024-04-22T06:31:12.030036+0000","last_fullsized":"2024-04-22T06:31:12.030036+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:30:54.947156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:30:54.947156+0000","last_clean_scrub_stamp":"2024-04-22T06:30:54.947156+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:32:37.920238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558994+0000","last_change":"2024-04-22T06:31:11.040152+0000","last_active":"2024-04-22T06:31:12.558994+0000","last_peered":"2024-04-22T06:31:12.558994+0000","last_clean":"2024-04-22T06:31:12.558994+0000","last_became_active":"2024-04-22T06:31:11.039749+0000","last_became_peered":"2024-04-22T06:31:11.039749+0000","last_unstale":"2024-04-22T06:31:12.558994+0000","last_undegraded":"2024-04-22T06:31:12.558994+0000","last_fullsized":"2024-04-22T06:31:12.558994+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:31:23.103876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031283+0000","last_change":"2024-04-22T06:31:07.011062+0000","last_active":"2024-04-22T06:31:12.031283+0000","last_peered":"2024-04-22T06:31:12.031283+0000","last_clean":"2024-04-22T06:31:12.031283+0000","last_became_active":"2024-04-22T06:31:07.010755+0000","last_became_peered":"2024-04-22T06:31:07.010755+0000","last_unstale":"2024-04-22T06:31:12.031283+0000","last_undegraded":"2024-04-22T06:31:12.031283+0000","last_fullsized":"2024-04-22T06:31:12.031283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:57:01.373036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560069+0000","last_change":"2024-04-22T06:31:05.998083+0000","last_active":"2024-04-22T06:31:12.560069+0000","last_peered":"2024-04-22T06:31:12.560069+0000","last_clean":"2024-04-22T06:31:12.560069+0000","last_became_active":"2024-04-22T06:31:05.997777+0000","last_became_peered":"2024-04-22T06:31:05.997777+0000","last_unstale":"2024-04-22T06:31:12.560069+0000","last_undegraded":"2024-04-22T06:31:12.560069+0000","last_fullsized":"2024-04-22T06:31:12.560069+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:15:13.171198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558666+0000","last_change":"2024-04-22T06:31:11.049607+0000","last_active":"2024-04-22T06:31:12.558666+0000","last_peered":"2024-04-22T06:31:12.558666+0000","last_clean":"2024-04-22T06:31:12.558666+0000","last_became_active":"2024-04-22T06:31:11.049156+0000","last_became_peered":"2024-04-22T06:31:11.049156+0000","last_unstale":"2024-04-22T06:31:12.558666+0000","last_undegraded":"2024-04-22T06:31:12.558666+0000","last_fullsized":"2024-04-22T06:31:12.558666+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:15:48.599278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.032032+0000","last_change":"2024-04-22T06:31:07.026306+0000","last_active":"2024-04-22T06:31:12.032032+0000","last_peered":"2024-04-22T06:31:12.032032+0000","last_clean":"2024-04-22T06:31:12.032032+0000","last_became_active":"2024-04-22T06:31:07.025987+0000","last_became_peered":"2024-04-22T06:31:07.025987+0000","last_unstale":"2024-04-22T06:31:12.032032+0000","last_undegraded":"2024-04-22T06:31:12.032032+0000","last_fullsized":"2024-04-22T06:31:12.032032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:50:31.803180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560942+0000","last_change":"2024-04-22T06:31:06.003115+0000","last_active":"2024-04-22T06:31:12.560942+0000","last_peered":"2024-04-22T06:31:12.560942+0000","last_clean":"2024-04-22T06:31:12.560942+0000","last_became_active":"2024-04-22T06:31:06.001945+0000","last_became_peered":"2024-04-22T06:31:06.001945+0000","last_unstale":"2024-04-22T06:31:12.560942+0000","last_undegraded":"2024-04-22T06:31:12.560942+0000","last_fullsized":"2024-04-22T06:31:12.560942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:59:25.770504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561105+0000","last_change":"2024-04-22T06:31:11.058841+0000","last_active":"2024-04-22T06:31:12.561105+0000","last_peered":"2024-04-22T06:31:12.561105+0000","last_clean":"2024-04-22T06:31:12.561105+0000","last_became_active":"2024-04-22T06:31:11.058407+0000","last_became_peered":"2024-04-22T06:31:11.058407+0000","last_unstale":"2024-04-22T06:31:12.561105+0000","last_undegraded":"2024-04-22T06:31:12.561105+0000","last_fullsized":"2024-04-22T06:31:12.561105+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:30:20.432210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031032+0000","last_change":"2024-04-22T06:31:07.022314+0000","last_active":"2024-04-22T06:31:12.031032+0000","last_peered":"2024-04-22T06:31:12.031032+0000","last_clean":"2024-04-22T06:31:12.031032+0000","last_became_active":"2024-04-22T06:31:07.021497+0000","last_became_peered":"2024-04-22T06:31:07.021497+0000","last_unstale":"2024-04-22T06:31:12.031032+0000","last_undegraded":"2024-04-22T06:31:12.031032+0000","last_fullsized":"2024-04-22T06:31:12.031032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:16:25.960359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559348+0000","last_change":"2024-04-22T06:31:06.003805+0000","last_active":"2024-04-22T06:31:12.559348+0000","last_peered":"2024-04-22T06:31:12.559348+0000","last_clean":"2024-04-22T06:31:12.559348+0000","last_became_active":"2024-04-22T06:31:06.002735+0000","last_became_peered":"2024-04-22T06:31:06.002735+0000","last_unstale":"2024-04-22T06:31:12.559348+0000","last_undegraded":"2024-04-22T06:31:12.559348+0000","last_fullsized":"2024-04-22T06:31:12.559348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:41:16.155628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030380+0000","last_change":"2024-04-22T06:31:11.043003+0000","last_active":"2024-04-22T06:31:12.030380+0000","last_peered":"2024-04-22T06:31:12.030380+0000","last_clean":"2024-04-22T06:31:12.030380+0000","last_became_active":"2024-04-22T06:31:11.042599+0000","last_became_peered":"2024-04-22T06:31:11.042599+0000","last_unstale":"2024-04-22T06:31:12.030380+0000","last_undegraded":"2024-04-22T06:31:12.030380+0000","last_fullsized":"2024-04-22T06:31:12.030380+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:28:58.949214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030520+0000","last_change":"2024-04-22T06:31:07.021258+0000","last_active":"2024-04-22T06:31:12.030520+0000","last_peered":"2024-04-22T06:31:12.030520+0000","last_clean":"2024-04-22T06:31:12.030520+0000","last_became_active":"2024-04-22T06:31:07.020976+0000","last_became_peered":"2024-04-22T06:31:07.020976+0000","last_unstale":"2024-04-22T06:31:12.030520+0000","last_undegraded":"2024-04-22T06:31:12.030520+0000","last_fullsized":"2024-04-22T06:31:12.030520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:19:33.377626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"24'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.033565+0000","last_change":"2024-04-22T06:31:06.002751+0000","last_active":"2024-04-22T06:31:12.033565+0000","last_peered":"2024-04-22T06:31:12.033565+0000","last_clean":"2024-04-22T06:31:12.033565+0000","last_became_active":"2024-04-22T06:31:06.002489+0000","last_became_peered":"2024-04-22T06:31:06.002489+0000","last_unstale":"2024-04-22T06:31:12.033565+0000","last_undegraded":"2024-04-22T06:31:12.033565+0000","last_fullsized":"2024-04-22T06:31:12.033565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:20:05.937529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.032720+0000","last_change":"2024-04-22T06:31:11.048954+0000","last_active":"2024-04-22T06:31:12.032720+0000","last_peered":"2024-04-22T06:31:12.032720+0000","last_clean":"2024-04-22T06:31:12.032720+0000","last_became_active":"2024-04-22T06:31:11.048214+0000","last_became_peered":"2024-04-22T06:31:11.048214+0000","last_unstale":"2024-04-22T06:31:12.032720+0000","last_undegraded":"2024-04-22T06:31:12.032720+0000","last_fullsized":"2024-04-22T06:31:12.032720+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:13:21.903203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030103+0000","last_change":"2024-04-22T06:31:07.021258+0000","last_active":"2024-04-22T06:31:12.030103+0000","last_peered":"2024-04-22T06:31:12.030103+0000","last_clean":"2024-04-22T06:31:12.030103+0000","last_became_active":"2024-04-22T06:31:07.021036+0000","last_became_peered":"2024-04-22T06:31:07.021036+0000","last_unstale":"2024-04-22T06:31:12.030103+0000","last_undegraded":"2024-04-22T06:31:12.030103+0000","last_fullsized":"2024-04-22T06:31:12.030103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:51:39.990228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"24'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559515+0000","last_change":"2024-04-22T06:31:05.998270+0000","last_active":"2024-04-22T06:31:12.559515+0000","last_peered":"2024-04-22T06:31:12.559515+0000","last_clean":"2024-04-22T06:31:12.559515+0000","last_became_active":"2024-04-22T06:31:05.997925+0000","last_became_peered":"2024-04-22T06:31:05.997925+0000","last_unstale":"2024-04-22T06:31:12.559515+0000","last_undegraded":"2024-04-22T06:31:12.559515+0000","last_fullsized":"2024-04-22T06:31:12.559515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:49:44.167932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560845+0000","last_change":"2024-04-22T06:31:11.051106+0000","last_active":"2024-04-22T06:31:12.560845+0000","last_peered":"2024-04-22T06:31:12.560845+0000","last_clean":"2024-04-22T06:31:12.560845+0000","last_became_active":"2024-04-22T06:31:11.050721+0000","last_became_peered":"2024-04-22T06:31:11.050721+0000","last_unstale":"2024-04-22T06:31:12.560845+0000","last_undegraded":"2024-04-22T06:31:12.560845+0000","last_fullsized":"2024-04-22T06:31:12.560845+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:58:43.521850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558584+0000","last_change":"2024-04-22T06:31:07.020281+0000","last_active":"2024-04-22T06:31:12.558584+0000","last_peered":"2024-04-22T06:31:12.558584+0000","last_clean":"2024-04-22T06:31:12.558584+0000","last_became_active":"2024-04-22T06:31:07.020012+0000","last_became_peered":"2024-04-22T06:31:07.020012+0000","last_unstale":"2024-04-22T06:31:12.558584+0000","last_undegraded":"2024-04-22T06:31:12.558584+0000","last_fullsized":"2024-04-22T06:31:12.558584+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:20:40.301631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558743+0000","last_change":"2024-04-22T06:31:05.999918+0000","last_active":"2024-04-22T06:31:12.558743+0000","last_peered":"2024-04-22T06:31:12.558743+0000","last_clean":"2024-04-22T06:31:12.558743+0000","last_became_active":"2024-04-22T06:31:05.999594+0000","last_became_peered":"2024-04-22T06:31:05.999594+0000","last_unstale":"2024-04-22T06:31:12.558743+0000","last_undegraded":"2024-04-22T06:31:12.558743+0000","last_fullsized":"2024-04-22T06:31:12.558743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:15:21.376196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559535+0000","last_change":"2024-04-22T06:31:11.055940+0000","last_active":"2024-04-22T06:31:12.559535+0000","last_peered":"2024-04-22T06:31:12.559535+0000","last_clean":"2024-04-22T06:31:12.559535+0000","last_became_active":"2024-04-22T06:31:11.054873+0000","last_became_peered":"2024-04-22T06:31:11.054873+0000","last_unstale":"2024-04-22T06:31:12.559535+0000","last_undegraded":"2024-04-22T06:31:12.559535+0000","last_fullsized":"2024-04-22T06:31:12.559535+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:13:51.061199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030857+0000","last_change":"2024-04-22T06:31:07.021839+0000","last_active":"2024-04-22T06:31:12.030857+0000","last_peered":"2024-04-22T06:31:12.030857+0000","last_clean":"2024-04-22T06:31:12.030857+0000","last_became_active":"2024-04-22T06:31:07.021585+0000","last_became_peered":"2024-04-22T06:31:07.021585+0000","last_unstale":"2024-04-22T06:31:12.030857+0000","last_undegraded":"2024-04-22T06:31:12.030857+0000","last_fullsized":"2024-04-22T06:31:12.030857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:55:15.275802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031584+0000","last_change":"2024-04-22T06:31:06.004058+0000","last_active":"2024-04-22T06:31:12.031584+0000","last_peered":"2024-04-22T06:31:12.031584+0000","last_clean":"2024-04-22T06:31:12.031584+0000","last_became_active":"2024-04-22T06:31:06.003791+0000","last_became_peered":"2024-04-22T06:31:06.003791+0000","last_unstale":"2024-04-22T06:31:12.031584+0000","last_undegraded":"2024-04-22T06:31:12.031584+0000","last_fullsized":"2024-04-22T06:31:12.031584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:03:08.913113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559058+0000","last_change":"2024-04-22T06:31:11.054245+0000","last_active":"2024-04-22T06:31:12.559058+0000","last_peered":"2024-04-22T06:31:12.559058+0000","last_clean":"2024-04-22T06:31:12.559058+0000","last_became_active":"2024-04-22T06:31:11.053551+0000","last_became_peered":"2024-04-22T06:31:11.053551+0000","last_unstale":"2024-04-22T06:31:12.559058+0000","last_undegraded":"2024-04-22T06:31:12.559058+0000","last_fullsized":"2024-04-22T06:31:12.559058+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:34:06.312700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559478+0000","last_change":"2024-04-22T06:31:07.023273+0000","last_active":"2024-04-22T06:31:12.559478+0000","last_peered":"2024-04-22T06:31:12.559478+0000","last_clean":"2024-04-22T06:31:12.559478+0000","last_became_active":"2024-04-22T06:31:07.022978+0000","last_became_peered":"2024-04-22T06:31:07.022978+0000","last_unstale":"2024-04-22T06:31:12.559478+0000","last_undegraded":"2024-04-22T06:31:12.559478+0000","last_fullsized":"2024-04-22T06:31:12.559478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:10:50.192298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"24'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561165+0000","last_change":"2024-04-22T06:31:06.005510+0000","last_active":"2024-04-22T06:31:12.561165+0000","last_peered":"2024-04-22T06:31:12.561165+0000","last_clean":"2024-04-22T06:31:12.561165+0000","last_became_active":"2024-04-22T06:31:06.005047+0000","last_became_peered":"2024-04-22T06:31:06.005047+0000","last_unstale":"2024-04-22T06:31:12.561165+0000","last_undegraded":"2024-04-22T06:31:12.561165+0000","last_fullsized":"2024-04-22T06:31:12.561165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:25:51.227631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.032100+0000","last_change":"2024-04-22T06:31:11.046209+0000","last_active":"2024-04-22T06:31:12.032100+0000","last_peered":"2024-04-22T06:31:12.032100+0000","last_clean":"2024-04-22T06:31:12.032100+0000","last_became_active":"2024-04-22T06:31:11.045339+0000","last_became_peered":"2024-04-22T06:31:11.045339+0000","last_unstale":"2024-04-22T06:31:12.032100+0000","last_undegraded":"2024-04-22T06:31:12.032100+0000","last_fullsized":"2024-04-22T06:31:12.032100+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:39:16.659010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031172+0000","last_change":"2024-04-22T06:31:07.025901+0000","last_active":"2024-04-22T06:31:12.031172+0000","last_peered":"2024-04-22T06:31:12.031172+0000","last_clean":"2024-04-22T06:31:12.031172+0000","last_became_active":"2024-04-22T06:31:07.025590+0000","last_became_peered":"2024-04-22T06:31:07.025590+0000","last_unstale":"2024-04-22T06:31:12.031172+0000","last_undegraded":"2024-04-22T06:31:12.031172+0000","last_fullsized":"2024-04-22T06:31:12.031172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:00:26.184054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:19.272923+0000","last_change":"2024-04-22T06:31:06.004609+0000","last_active":"2024-04-22T06:31:19.272923+0000","last_peered":"2024-04-22T06:31:19.272923+0000","last_clean":"2024-04-22T06:31:19.272923+0000","last_became_active":"2024-04-22T06:31:06.004319+0000","last_became_peered":"2024-04-22T06:31:06.004319+0000","last_unstale":"2024-04-22T06:31:19.272923+0000","last_undegraded":"2024-04-22T06:31:19.272923+0000","last_fullsized":"2024-04-22T06:31:19.272923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:32:47.198906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561266+0000","last_change":"2024-04-22T06:31:11.054706+0000","last_active":"2024-04-22T06:31:12.561266+0000","last_peered":"2024-04-22T06:31:12.561266+0000","last_clean":"2024-04-22T06:31:12.561266+0000","last_became_active":"2024-04-22T06:31:11.054307+0000","last_became_peered":"2024-04-22T06:31:11.054307+0000","last_unstale":"2024-04-22T06:31:12.561266+0000","last_undegraded":"2024-04-22T06:31:12.561266+0000","last_fullsized":"2024-04-22T06:31:12.561266+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:05:02.465114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.033403+0000","last_change":"2024-04-22T06:31:07.020739+0000","last_active":"2024-04-22T06:31:12.033403+0000","last_peered":"2024-04-22T06:31:12.033403+0000","last_clean":"2024-04-22T06:31:12.033403+0000","last_became_active":"2024-04-22T06:31:07.020070+0000","last_became_peered":"2024-04-22T06:31:07.020070+0000","last_unstale":"2024-04-22T06:31:12.033403+0000","last_undegraded":"2024-04-22T06:31:12.033403+0000","last_fullsized":"2024-04-22T06:31:12.033403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:50:22.102753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"24'3","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560805+0000","last_change":"2024-04-22T06:31:06.003733+0000","last_active":"2024-04-22T06:31:12.560805+0000","last_peered":"2024-04-22T06:31:12.560805+0000","last_clean":"2024-04-22T06:31:12.560805+0000","last_became_active":"2024-04-22T06:31:06.003465+0000","last_became_peered":"2024-04-22T06:31:06.003465+0000","last_unstale":"2024-04-22T06:31:12.560805+0000","last_undegraded":"2024-04-22T06:31:12.560805+0000","last_fullsized":"2024-04-22T06:31:12.560805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:19:36.023311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.032875+0000","last_change":"2024-04-22T06:31:11.058041+0000","last_active":"2024-04-22T06:31:12.032875+0000","last_peered":"2024-04-22T06:31:12.032875+0000","last_clean":"2024-04-22T06:31:12.032875+0000","last_became_active":"2024-04-22T06:31:11.057243+0000","last_became_peered":"2024-04-22T06:31:11.057243+0000","last_unstale":"2024-04-22T06:31:12.032875+0000","last_undegraded":"2024-04-22T06:31:12.032875+0000","last_fullsized":"2024-04-22T06:31:12.032875+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:49:06.781140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560767+0000","last_change":"2024-04-22T06:31:07.015369+0000","last_active":"2024-04-22T06:31:12.560767+0000","last_peered":"2024-04-22T06:31:12.560767+0000","last_clean":"2024-04-22T06:31:12.560767+0000","last_became_active":"2024-04-22T06:31:07.015088+0000","last_became_peered":"2024-04-22T06:31:07.015088+0000","last_unstale":"2024-04-22T06:31:12.560767+0000","last_undegraded":"2024-04-22T06:31:12.560767+0000","last_fullsized":"2024-04-22T06:31:12.560767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:10:09.449507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030766+0000","last_change":"2024-04-22T06:31:05.999636+0000","last_active":"2024-04-22T06:31:12.030766+0000","last_peered":"2024-04-22T06:31:12.030766+0000","last_clean":"2024-04-22T06:31:12.030766+0000","last_became_active":"2024-04-22T06:31:05.999329+0000","last_became_peered":"2024-04-22T06:31:05.999329+0000","last_unstale":"2024-04-22T06:31:12.030766+0000","last_undegraded":"2024-04-22T06:31:12.030766+0000","last_fullsized":"2024-04-22T06:31:12.030766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:09:06.670827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559595+0000","last_change":"2024-04-22T06:31:11.053530+0000","last_active":"2024-04-22T06:31:12.559595+0000","last_peered":"2024-04-22T06:31:12.559595+0000","last_clean":"2024-04-22T06:31:12.559595+0000","last_became_active":"2024-04-22T06:31:11.053100+0000","last_became_peered":"2024-04-22T06:31:11.053100+0000","last_unstale":"2024-04-22T06:31:12.559595+0000","last_undegraded":"2024-04-22T06:31:12.559595+0000","last_fullsized":"2024-04-22T06:31:12.559595+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:57:16.506840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030657+0000","last_change":"2024-04-22T06:31:07.015838+0000","last_active":"2024-04-22T06:31:12.030657+0000","last_peered":"2024-04-22T06:31:12.030657+0000","last_clean":"2024-04-22T06:31:12.030657+0000","last_became_active":"2024-04-22T06:31:07.015545+0000","last_became_peered":"2024-04-22T06:31:07.015545+0000","last_unstale":"2024-04-22T06:31:12.030657+0000","last_undegraded":"2024-04-22T06:31:12.030657+0000","last_fullsized":"2024-04-22T06:31:12.030657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:43:54.251061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559092+0000","last_change":"2024-04-22T06:31:06.005442+0000","last_active":"2024-04-22T06:31:12.559092+0000","last_peered":"2024-04-22T06:31:12.559092+0000","last_clean":"2024-04-22T06:31:12.559092+0000","last_became_active":"2024-04-22T06:31:06.005156+0000","last_became_peered":"2024-04-22T06:31:06.005156+0000","last_unstale":"2024-04-22T06:31:12.559092+0000","last_undegraded":"2024-04-22T06:31:12.559092+0000","last_fullsized":"2024-04-22T06:31:12.559092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:52:32.138519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560401+0000","last_change":"2024-04-22T06:31:11.046885+0000","last_active":"2024-04-22T06:31:12.560401+0000","last_peered":"2024-04-22T06:31:12.560401+0000","last_clean":"2024-04-22T06:31:12.560401+0000","last_became_active":"2024-04-22T06:31:11.046444+0000","last_became_peered":"2024-04-22T06:31:11.046444+0000","last_unstale":"2024-04-22T06:31:12.560401+0000","last_undegraded":"2024-04-22T06:31:12.560401+0000","last_fullsized":"2024-04-22T06:31:12.560401+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:17:54.523416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560687+0000","last_change":"2024-04-22T06:31:11.061784+0000","last_active":"2024-04-22T06:31:12.560687+0000","last_peered":"2024-04-22T06:31:12.560687+0000","last_clean":"2024-04-22T06:31:12.560687+0000","last_became_active":"2024-04-22T06:31:11.061382+0000","last_became_peered":"2024-04-22T06:31:11.061382+0000","last_unstale":"2024-04-22T06:31:12.560687+0000","last_undegraded":"2024-04-22T06:31:12.560687+0000","last_fullsized":"2024-04-22T06:31:12.560687+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:12:53.829206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560353+0000","last_change":"2024-04-22T06:31:07.023542+0000","last_active":"2024-04-22T06:31:12.560353+0000","last_peered":"2024-04-22T06:31:12.560353+0000","last_clean":"2024-04-22T06:31:12.560353+0000","last_became_active":"2024-04-22T06:31:07.023240+0000","last_became_peered":"2024-04-22T06:31:07.023240+0000","last_unstale":"2024-04-22T06:31:12.560353+0000","last_undegraded":"2024-04-22T06:31:12.560353+0000","last_fullsized":"2024-04-22T06:31:12.560353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:15:34.727195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030673+0000","last_change":"2024-04-22T06:31:06.003080+0000","last_active":"2024-04-22T06:31:12.030673+0000","last_peered":"2024-04-22T06:31:12.030673+0000","last_clean":"2024-04-22T06:31:12.030673+0000","last_became_active":"2024-04-22T06:31:06.002725+0000","last_became_peered":"2024-04-22T06:31:06.002725+0000","last_unstale":"2024-04-22T06:31:12.030673+0000","last_undegraded":"2024-04-22T06:31:12.030673+0000","last_fullsized":"2024-04-22T06:31:12.030673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:46:38.249137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559100+0000","last_change":"2024-04-22T06:31:11.047782+0000","last_active":"2024-04-22T06:31:12.559100+0000","last_peered":"2024-04-22T06:31:12.559100+0000","last_clean":"2024-04-22T06:31:12.559100+0000","last_became_active":"2024-04-22T06:31:11.047407+0000","last_became_peered":"2024-04-22T06:31:11.047407+0000","last_unstale":"2024-04-22T06:31:12.559100+0000","last_undegraded":"2024-04-22T06:31:12.559100+0000","last_fullsized":"2024-04-22T06:31:12.559100+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:48:05.416994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559643+0000","last_change":"2024-04-22T06:31:07.025799+0000","last_active":"2024-04-22T06:31:12.559643+0000","last_peered":"2024-04-22T06:31:12.559643+0000","last_clean":"2024-04-22T06:31:12.559643+0000","last_became_active":"2024-04-22T06:31:07.025420+0000","last_became_peered":"2024-04-22T06:31:07.025420+0000","last_unstale":"2024-04-22T06:31:12.559643+0000","last_undegraded":"2024-04-22T06:31:12.559643+0000","last_fullsized":"2024-04-22T06:31:12.559643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:39:40.107621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560547+0000","last_change":"2024-04-22T06:31:06.005805+0000","last_active":"2024-04-22T06:31:12.560547+0000","last_peered":"2024-04-22T06:31:12.560547+0000","last_clean":"2024-04-22T06:31:12.560547+0000","last_became_active":"2024-04-22T06:31:06.005479+0000","last_became_peered":"2024-04-22T06:31:06.005479+0000","last_unstale":"2024-04-22T06:31:12.560547+0000","last_undegraded":"2024-04-22T06:31:12.560547+0000","last_fullsized":"2024-04-22T06:31:12.560547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:59:37.140612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"24'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558740+0000","last_change":"2024-04-22T06:31:05.998354+0000","last_active":"2024-04-22T06:31:12.558740+0000","last_peered":"2024-04-22T06:31:12.558740+0000","last_clean":"2024-04-22T06:31:12.558740+0000","last_became_active":"2024-04-22T06:31:05.998071+0000","last_became_peered":"2024-04-22T06:31:05.998071+0000","last_unstale":"2024-04-22T06:31:12.558740+0000","last_undegraded":"2024-04-22T06:31:12.558740+0000","last_fullsized":"2024-04-22T06:31:12.558740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:12:13.343435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561078+0000","last_change":"2024-04-22T06:31:07.024109+0000","last_active":"2024-04-22T06:31:12.561078+0000","last_peered":"2024-04-22T06:31:12.561078+0000","last_clean":"2024-04-22T06:31:12.561078+0000","last_became_active":"2024-04-22T06:31:07.023346+0000","last_became_peered":"2024-04-22T06:31:07.023346+0000","last_unstale":"2024-04-22T06:31:12.561078+0000","last_undegraded":"2024-04-22T06:31:12.561078+0000","last_fullsized":"2024-04-22T06:31:12.561078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:02:42.662312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559888+0000","last_change":"2024-04-22T06:31:11.047132+0000","last_active":"2024-04-22T06:31:12.559888+0000","last_peered":"2024-04-22T06:31:12.559888+0000","last_clean":"2024-04-22T06:31:12.559888+0000","last_became_active":"2024-04-22T06:31:11.046671+0000","last_became_peered":"2024-04-22T06:31:11.046671+0000","last_unstale":"2024-04-22T06:31:12.559888+0000","last_undegraded":"2024-04-22T06:31:12.559888+0000","last_fullsized":"2024-04-22T06:31:12.559888+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:41:57.706735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"24'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561258+0000","last_change":"2024-04-22T06:31:06.004971+0000","last_active":"2024-04-22T06:31:12.561258+0000","last_peered":"2024-04-22T06:31:12.561258+0000","last_clean":"2024-04-22T06:31:12.561258+0000","last_became_active":"2024-04-22T06:31:06.004473+0000","last_became_peered":"2024-04-22T06:31:06.004473+0000","last_unstale":"2024-04-22T06:31:12.561258+0000","last_undegraded":"2024-04-22T06:31:12.561258+0000","last_fullsized":"2024-04-22T06:31:12.561258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:47:11.735311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.032950+0000","last_change":"2024-04-22T06:31:07.018314+0000","last_active":"2024-04-22T06:31:12.032950+0000","last_peered":"2024-04-22T06:31:12.032950+0000","last_clean":"2024-04-22T06:31:12.032950+0000","last_became_active":"2024-04-22T06:31:07.018033+0000","last_became_peered":"2024-04-22T06:31:07.018033+0000","last_unstale":"2024-04-22T06:31:12.032950+0000","last_undegraded":"2024-04-22T06:31:12.032950+0000","last_fullsized":"2024-04-22T06:31:12.032950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:06:06.099494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559988+0000","last_change":"2024-04-22T06:31:11.055874+0000","last_active":"2024-04-22T06:31:12.559988+0000","last_peered":"2024-04-22T06:31:12.559988+0000","last_clean":"2024-04-22T06:31:12.559988+0000","last_became_active":"2024-04-22T06:31:11.055371+0000","last_became_peered":"2024-04-22T06:31:11.055371+0000","last_unstale":"2024-04-22T06:31:12.559988+0000","last_undegraded":"2024-04-22T06:31:12.559988+0000","last_fullsized":"2024-04-22T06:31:12.559988+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:06:03.909337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558461+0000","last_change":"2024-04-22T06:31:07.024780+0000","last_active":"2024-04-22T06:31:12.558461+0000","last_peered":"2024-04-22T06:31:12.558461+0000","last_clean":"2024-04-22T06:31:12.558461+0000","last_became_active":"2024-04-22T06:31:07.024501+0000","last_became_peered":"2024-04-22T06:31:07.024501+0000","last_unstale":"2024-04-22T06:31:12.558461+0000","last_undegraded":"2024-04-22T06:31:12.558461+0000","last_fullsized":"2024-04-22T06:31:12.558461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:12:45.981232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558747+0000","last_change":"2024-04-22T06:31:07.014546+0000","last_active":"2024-04-22T06:31:12.558747+0000","last_peered":"2024-04-22T06:31:12.558747+0000","last_clean":"2024-04-22T06:31:12.558747+0000","last_became_active":"2024-04-22T06:31:07.014196+0000","last_became_peered":"2024-04-22T06:31:07.014196+0000","last_unstale":"2024-04-22T06:31:12.558747+0000","last_undegraded":"2024-04-22T06:31:12.558747+0000","last_fullsized":"2024-04-22T06:31:12.558747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:57:33.136875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031490+0000","last_change":"2024-04-22T06:31:07.018563+0000","last_active":"2024-04-22T06:31:12.031490+0000","last_peered":"2024-04-22T06:31:12.031490+0000","last_clean":"2024-04-22T06:31:12.031490+0000","last_became_active":"2024-04-22T06:31:07.018289+0000","last_became_peered":"2024-04-22T06:31:07.018289+0000","last_unstale":"2024-04-22T06:31:12.031490+0000","last_undegraded":"2024-04-22T06:31:12.031490+0000","last_fullsized":"2024-04-22T06:31:12.031490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:30:11.892544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560983+0000","last_change":"2024-04-22T06:31:07.022901+0000","last_active":"2024-04-22T06:31:12.560983+0000","last_peered":"2024-04-22T06:31:12.560983+0000","last_clean":"2024-04-22T06:31:12.560983+0000","last_became_active":"2024-04-22T06:31:07.022420+0000","last_became_peered":"2024-04-22T06:31:07.022420+0000","last_unstale":"2024-04-22T06:31:12.560983+0000","last_undegraded":"2024-04-22T06:31:12.560983+0000","last_fullsized":"2024-04-22T06:31:12.560983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:24:18.929896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559253+0000","last_change":"2024-04-22T06:31:07.022875+0000","last_active":"2024-04-22T06:31:12.559253+0000","last_peered":"2024-04-22T06:31:12.559253+0000","last_clean":"2024-04-22T06:31:12.559253+0000","last_became_active":"2024-04-22T06:31:07.022545+0000","last_became_peered":"2024-04-22T06:31:07.022545+0000","last_unstale":"2024-04-22T06:31:12.559253+0000","last_undegraded":"2024-04-22T06:31:12.559253+0000","last_fullsized":"2024-04-22T06:31:12.559253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:02:30.464508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560354+0000","last_change":"2024-04-22T06:31:07.026367+0000","last_active":"2024-04-22T06:31:12.560354+0000","last_peered":"2024-04-22T06:31:12.560354+0000","last_clean":"2024-04-22T06:31:12.560354+0000","last_became_active":"2024-04-22T06:31:07.026085+0000","last_became_peered":"2024-04-22T06:31:07.026085+0000","last_unstale":"2024-04-22T06:31:12.560354+0000","last_undegraded":"2024-04-22T06:31:12.560354+0000","last_fullsized":"2024-04-22T06:31:12.560354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:02:55.368591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559243+0000","last_change":"2024-04-22T06:31:07.022989+0000","last_active":"2024-04-22T06:31:12.559243+0000","last_peered":"2024-04-22T06:31:12.559243+0000","last_clean":"2024-04-22T06:31:12.559243+0000","last_became_active":"2024-04-22T06:31:07.022688+0000","last_became_peered":"2024-04-22T06:31:07.022688+0000","last_unstale":"2024-04-22T06:31:12.559243+0000","last_undegraded":"2024-04-22T06:31:12.559243+0000","last_fullsized":"2024-04-22T06:31:12.559243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:24:10.910731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559590+0000","last_change":"2024-04-22T06:31:07.026315+0000","last_active":"2024-04-22T06:31:12.559590+0000","last_peered":"2024-04-22T06:31:12.559590+0000","last_clean":"2024-04-22T06:31:12.559590+0000","last_became_active":"2024-04-22T06:31:07.025951+0000","last_became_peered":"2024-04-22T06:31:07.025951+0000","last_unstale":"2024-04-22T06:31:12.559590+0000","last_undegraded":"2024-04-22T06:31:12.559590+0000","last_fullsized":"2024-04-22T06:31:12.559590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:54:12.160058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030585+0000","last_change":"2024-04-22T06:31:07.022748+0000","last_active":"2024-04-22T06:31:12.030585+0000","last_peered":"2024-04-22T06:31:12.030585+0000","last_clean":"2024-04-22T06:31:12.030585+0000","last_became_active":"2024-04-22T06:31:07.022481+0000","last_became_peered":"2024-04-22T06:31:07.022481+0000","last_unstale":"2024-04-22T06:31:12.030585+0000","last_undegraded":"2024-04-22T06:31:12.030585+0000","last_fullsized":"2024-04-22T06:31:12.030585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:56:32.631637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560456+0000","last_change":"2024-04-22T06:31:07.021939+0000","last_active":"2024-04-22T06:31:12.560456+0000","last_peered":"2024-04-22T06:31:12.560456+0000","last_clean":"2024-04-22T06:31:12.560456+0000","last_became_active":"2024-04-22T06:31:07.021638+0000","last_became_peered":"2024-04-22T06:31:07.021638+0000","last_unstale":"2024-04-22T06:31:12.560456+0000","last_undegraded":"2024-04-22T06:31:12.560456+0000","last_fullsized":"2024-04-22T06:31:12.560456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:05:27.106092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031390+0000","last_change":"2024-04-22T06:31:07.015844+0000","last_active":"2024-04-22T06:31:12.031390+0000","last_peered":"2024-04-22T06:31:12.031390+0000","last_clean":"2024-04-22T06:31:12.031390+0000","last_became_active":"2024-04-22T06:31:07.015520+0000","last_became_peered":"2024-04-22T06:31:07.015520+0000","last_unstale":"2024-04-22T06:31:12.031390+0000","last_undegraded":"2024-04-22T06:31:12.031390+0000","last_fullsized":"2024-04-22T06:31:12.031390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:24:04.676892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560128+0000","last_change":"2024-04-22T06:31:07.023697+0000","last_active":"2024-04-22T06:31:12.560128+0000","last_peered":"2024-04-22T06:31:12.560128+0000","last_clean":"2024-04-22T06:31:12.560128+0000","last_became_active":"2024-04-22T06:31:07.023283+0000","last_became_peered":"2024-04-22T06:31:07.023283+0000","last_unstale":"2024-04-22T06:31:12.560128+0000","last_undegraded":"2024-04-22T06:31:12.560128+0000","last_fullsized":"2024-04-22T06:31:12.560128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:43:01.338104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558868+0000","last_change":"2024-04-22T06:31:07.023953+0000","last_active":"2024-04-22T06:31:12.558868+0000","last_peered":"2024-04-22T06:31:12.558868+0000","last_clean":"2024-04-22T06:31:12.558868+0000","last_became_active":"2024-04-22T06:31:07.023704+0000","last_became_peered":"2024-04-22T06:31:07.023704+0000","last_unstale":"2024-04-22T06:31:12.558868+0000","last_undegraded":"2024-04-22T06:31:12.558868+0000","last_fullsized":"2024-04-22T06:31:12.558868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:59:32.219492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031619+0000","last_change":"2024-04-22T06:31:07.027418+0000","last_active":"2024-04-22T06:31:12.031619+0000","last_peered":"2024-04-22T06:31:12.031619+0000","last_clean":"2024-04-22T06:31:12.031619+0000","last_became_active":"2024-04-22T06:31:07.027113+0000","last_became_peered":"2024-04-22T06:31:07.027113+0000","last_unstale":"2024-04-22T06:31:12.031619+0000","last_undegraded":"2024-04-22T06:31:12.031619+0000","last_fullsized":"2024-04-22T06:31:12.031619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:08:42.204674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559261+0000","last_change":"2024-04-22T06:31:07.015520+0000","last_active":"2024-04-22T06:31:12.559261+0000","last_peered":"2024-04-22T06:31:12.559261+0000","last_clean":"2024-04-22T06:31:12.559261+0000","last_became_active":"2024-04-22T06:31:07.015199+0000","last_became_peered":"2024-04-22T06:31:07.015199+0000","last_unstale":"2024-04-22T06:31:12.559261+0000","last_undegraded":"2024-04-22T06:31:12.559261+0000","last_fullsized":"2024-04-22T06:31:12.559261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:48:40.688931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031281+0000","last_change":"2024-04-22T06:31:07.016457+0000","last_active":"2024-04-22T06:31:12.031281+0000","last_peered":"2024-04-22T06:31:12.031281+0000","last_clean":"2024-04-22T06:31:12.031281+0000","last_became_active":"2024-04-22T06:31:07.016082+0000","last_became_peered":"2024-04-22T06:31:07.016082+0000","last_unstale":"2024-04-22T06:31:12.031281+0000","last_undegraded":"2024-04-22T06:31:12.031281+0000","last_fullsized":"2024-04-22T06:31:12.031281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:12:28.517065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030497+0000","last_change":"2024-04-22T06:31:07.019955+0000","last_active":"2024-04-22T06:31:12.030497+0000","last_peered":"2024-04-22T06:31:12.030497+0000","last_clean":"2024-04-22T06:31:12.030497+0000","last_became_active":"2024-04-22T06:31:07.019174+0000","last_became_peered":"2024-04-22T06:31:07.019174+0000","last_unstale":"2024-04-22T06:31:12.030497+0000","last_undegraded":"2024-04-22T06:31:12.030497+0000","last_fullsized":"2024-04-22T06:31:12.030497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:09:07.106161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560474+0000","last_change":"2024-04-22T06:31:07.019341+0000","last_active":"2024-04-22T06:31:12.560474+0000","last_peered":"2024-04-22T06:31:12.560474+0000","last_clean":"2024-04-22T06:31:12.560474+0000","last_became_active":"2024-04-22T06:31:07.019048+0000","last_became_peered":"2024-04-22T06:31:07.019048+0000","last_unstale":"2024-04-22T06:31:12.560474+0000","last_undegraded":"2024-04-22T06:31:12.560474+0000","last_fullsized":"2024-04-22T06:31:12.560474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:04:19.362686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559337+0000","last_change":"2024-04-22T06:31:07.019433+0000","last_active":"2024-04-22T06:31:12.559337+0000","last_peered":"2024-04-22T06:31:12.559337+0000","last_clean":"2024-04-22T06:31:12.559337+0000","last_became_active":"2024-04-22T06:31:07.019133+0000","last_became_peered":"2024-04-22T06:31:07.019133+0000","last_unstale":"2024-04-22T06:31:12.559337+0000","last_undegraded":"2024-04-22T06:31:12.559337+0000","last_fullsized":"2024-04-22T06:31:12.559337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:01:54.466486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561087+0000","last_change":"2024-04-22T06:31:07.023014+0000","last_active":"2024-04-22T06:31:12.561087+0000","last_peered":"2024-04-22T06:31:12.561087+0000","last_clean":"2024-04-22T06:31:12.561087+0000","last_became_active":"2024-04-22T06:31:07.022716+0000","last_became_peered":"2024-04-22T06:31:07.022716+0000","last_unstale":"2024-04-22T06:31:12.561087+0000","last_undegraded":"2024-04-22T06:31:12.561087+0000","last_fullsized":"2024-04-22T06:31:12.561087+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:11:47.309706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.033363+0000","last_change":"2024-04-22T06:31:07.020446+0000","last_active":"2024-04-22T06:31:12.033363+0000","last_peered":"2024-04-22T06:31:12.033363+0000","last_clean":"2024-04-22T06:31:12.033363+0000","last_became_active":"2024-04-22T06:31:07.018995+0000","last_became_peered":"2024-04-22T06:31:07.018995+0000","last_unstale":"2024-04-22T06:31:12.033363+0000","last_undegraded":"2024-04-22T06:31:12.033363+0000","last_fullsized":"2024-04-22T06:31:12.033363+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:56:45.031262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558848+0000","last_change":"2024-04-22T06:31:07.014927+0000","last_active":"2024-04-22T06:31:12.558848+0000","last_peered":"2024-04-22T06:31:12.558848+0000","last_clean":"2024-04-22T06:31:12.558848+0000","last_became_active":"2024-04-22T06:31:07.014652+0000","last_became_peered":"2024-04-22T06:31:07.014652+0000","last_unstale":"2024-04-22T06:31:12.558848+0000","last_undegraded":"2024-04-22T06:31:12.558848+0000","last_fullsized":"2024-04-22T06:31:12.558848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:51:33.506667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559161+0000","last_change":"2024-04-22T06:31:07.014209+0000","last_active":"2024-04-22T06:31:12.559161+0000","last_peered":"2024-04-22T06:31:12.559161+0000","last_clean":"2024-04-22T06:31:12.559161+0000","last_became_active":"2024-04-22T06:31:07.013946+0000","last_became_peered":"2024-04-22T06:31:07.013946+0000","last_unstale":"2024-04-22T06:31:12.559161+0000","last_undegraded":"2024-04-22T06:31:12.559161+0000","last_fullsized":"2024-04-22T06:31:12.559161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:40:31.669158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031004+0000","last_change":"2024-04-22T06:31:07.008764+0000","last_active":"2024-04-22T06:31:12.031004+0000","last_peered":"2024-04-22T06:31:12.031004+0000","last_clean":"2024-04-22T06:31:12.031004+0000","last_became_active":"2024-04-22T06:31:07.007810+0000","last_became_peered":"2024-04-22T06:31:07.007810+0000","last_unstale":"2024-04-22T06:31:12.031004+0000","last_undegraded":"2024-04-22T06:31:12.031004+0000","last_fullsized":"2024-04-22T06:31:12.031004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:49:54.894818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559408+0000","last_change":"2024-04-22T06:31:07.022141+0000","last_active":"2024-04-22T06:31:12.559408+0000","last_peered":"2024-04-22T06:31:12.559408+0000","last_clean":"2024-04-22T06:31:12.559408+0000","last_became_active":"2024-04-22T06:31:07.021845+0000","last_became_peered":"2024-04-22T06:31:07.021845+0000","last_unstale":"2024-04-22T06:31:12.559408+0000","last_undegraded":"2024-04-22T06:31:12.559408+0000","last_fullsized":"2024-04-22T06:31:12.559408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:26:28.598104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.029984+0000","last_change":"2024-04-22T06:31:07.012188+0000","last_active":"2024-04-22T06:31:12.029984+0000","last_peered":"2024-04-22T06:31:12.029984+0000","last_clean":"2024-04-22T06:31:12.029984+0000","last_became_active":"2024-04-22T06:31:07.011909+0000","last_became_peered":"2024-04-22T06:31:07.011909+0000","last_unstale":"2024-04-22T06:31:12.029984+0000","last_undegraded":"2024-04-22T06:31:12.029984+0000","last_fullsized":"2024-04-22T06:31:12.029984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:25:13.089928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031293+0000","last_change":"2024-04-22T06:31:07.020033+0000","last_active":"2024-04-22T06:31:12.031293+0000","last_peered":"2024-04-22T06:31:12.031293+0000","last_clean":"2024-04-22T06:31:12.031293+0000","last_became_active":"2024-04-22T06:31:07.019689+0000","last_became_peered":"2024-04-22T06:31:07.019689+0000","last_unstale":"2024-04-22T06:31:12.031293+0000","last_undegraded":"2024-04-22T06:31:12.031293+0000","last_fullsized":"2024-04-22T06:31:12.031293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:09:07.209203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030666+0000","last_change":"2024-04-22T06:31:07.027287+0000","last_active":"2024-04-22T06:31:12.030666+0000","last_peered":"2024-04-22T06:31:12.030666+0000","last_clean":"2024-04-22T06:31:12.030666+0000","last_became_active":"2024-04-22T06:31:07.027015+0000","last_became_peered":"2024-04-22T06:31:07.027015+0000","last_unstale":"2024-04-22T06:31:12.030666+0000","last_undegraded":"2024-04-22T06:31:12.030666+0000","last_fullsized":"2024-04-22T06:31:12.030666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:42:49.322413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031959+0000","last_change":"2024-04-22T06:31:07.026332+0000","last_active":"2024-04-22T06:31:12.031959+0000","last_peered":"2024-04-22T06:31:12.031959+0000","last_clean":"2024-04-22T06:31:12.031959+0000","last_became_active":"2024-04-22T06:31:07.026018+0000","last_became_peered":"2024-04-22T06:31:07.026018+0000","last_unstale":"2024-04-22T06:31:12.031959+0000","last_undegraded":"2024-04-22T06:31:12.031959+0000","last_fullsized":"2024-04-22T06:31:12.031959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:27:08.560056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560402+0000","last_change":"2024-04-22T06:31:07.020497+0000","last_active":"2024-04-22T06:31:12.560402+0000","last_peered":"2024-04-22T06:31:12.560402+0000","last_clean":"2024-04-22T06:31:12.560402+0000","last_became_active":"2024-04-22T06:31:07.020194+0000","last_became_peered":"2024-04-22T06:31:07.020194+0000","last_unstale":"2024-04-22T06:31:12.560402+0000","last_undegraded":"2024-04-22T06:31:12.560402+0000","last_fullsized":"2024-04-22T06:31:12.560402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:56:59.765295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560664+0000","last_change":"2024-04-22T06:31:07.018396+0000","last_active":"2024-04-22T06:31:12.560664+0000","last_peered":"2024-04-22T06:31:12.560664+0000","last_clean":"2024-04-22T06:31:12.560664+0000","last_became_active":"2024-04-22T06:31:07.018073+0000","last_became_peered":"2024-04-22T06:31:07.018073+0000","last_unstale":"2024-04-22T06:31:12.560664+0000","last_undegraded":"2024-04-22T06:31:12.560664+0000","last_fullsized":"2024-04-22T06:31:12.560664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:37:06.501499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030917+0000","last_change":"2024-04-22T06:31:07.014439+0000","last_active":"2024-04-22T06:31:12.030917+0000","last_peered":"2024-04-22T06:31:12.030917+0000","last_clean":"2024-04-22T06:31:12.030917+0000","last_became_active":"2024-04-22T06:31:07.014124+0000","last_became_peered":"2024-04-22T06:31:07.014124+0000","last_unstale":"2024-04-22T06:31:12.030917+0000","last_undegraded":"2024-04-22T06:31:12.030917+0000","last_fullsized":"2024-04-22T06:31:12.030917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:58:46.075068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":864256,"data_stored":695877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":270336,"data_stored":105017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":278528,"data_stored":105053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":104927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":274432,"data_stored":105087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":872448,"data_stored":695933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T06:31:35.453 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-22T06:31:35.454 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-22T06:31:35.454 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-22T06:31:35.454 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-22T06:31:35.454 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-22T06:31:35.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.589+0000 7fdf75bdb640 1 Processor -- start 2024-04-22T06:31:35.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.589+0000 7fdf75bdb640 1 -- start start 2024-04-22T06:31:35.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.590+0000 7fdf75bdb640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf701552a0 0x7fdf7015bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:35.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.590+0000 7fdf75bdb640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf7006c6e0 con 0x7fdf70155680 2024-04-22T06:31:35.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.590+0000 7fdf75bdb640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf7006c9c0 con 0x7fdf700cc4e0 2024-04-22T06:31:35.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.590+0000 7fdf75bdb640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdf700cb990 con 0x7fdf701552a0 2024-04-22T06:31:35.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.590+0000 7fdf6effd640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fdf70155680 0x7fdf700cbdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:49418/0 (socket says 172.21.15.44:49418) 2024-04-22T06:31:35.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.590+0000 7fdf6effd640 1 -- 172.21.15.44:0/2522294716 learned_addr learned my addr 172.21.15.44:0/2522294716 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:35.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.590+0000 7fdf6f7fe640 1 --2- 172.21.15.44:0/2522294716 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf701552a0 0x7fdf7015bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:35.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.590+0000 7fdf6e7fc640 1 -- 172.21.15.44:0/2522294716 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4289312217 0 0) 0x7fdf7006c6e0 con 0x7fdf70155680 2024-04-22T06:31:35.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.590+0000 7fdf6e7fc640 1 -- 172.21.15.44:0/2522294716 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf58003520 con 0x7fdf70155680 2024-04-22T06:31:35.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.591+0000 7fdf6e7fc640 1 -- 172.21.15.44:0/2522294716 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1684588415 0 0) 0x7fdf7006c9c0 con 0x7fdf700cc4e0 2024-04-22T06:31:35.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.591+0000 7fdf6e7fc640 1 -- 172.21.15.44:0/2522294716 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf7006c6e0 con 0x7fdf700cc4e0 2024-04-22T06:31:35.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.591+0000 7fdf6f7fe640 1 -- 172.21.15.44:0/2522294716 >> v1:172.21.15.70:6789/0 conn(0x7fdf700cc4e0 legacy=0x7fdf700ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:35.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.591+0000 7fdf6f7fe640 1 -- 172.21.15.44:0/2522294716 >> v1:172.21.15.44:6789/0 conn(0x7fdf70155680 legacy=0x7fdf700cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:35.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.591+0000 7fdf6f7fe640 1 -- 172.21.15.44:0/2522294716 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf7006cca0 con 0x7fdf701552a0 2024-04-22T06:31:35.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.591+0000 7fdf6f7fe640 1 --2- 172.21.15.44:0/2522294716 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf701552a0 0x7fdf7015bb80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fdf600097d0 tx=0x7fdf6002b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e37dcbb921a049f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:35.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.592+0000 7fdf6e7fc640 1 -- 172.21.15.44:0/2522294716 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf60034070 con 0x7fdf701552a0 2024-04-22T06:31:35.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.592+0000 7fdf6e7fc640 1 -- 172.21.15.44:0/2522294716 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf60030880 con 0x7fdf701552a0 2024-04-22T06:31:35.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.592+0000 7fdf6e7fc640 1 -- 172.21.15.44:0/2522294716 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf60030c50 con 0x7fdf701552a0 2024-04-22T06:31:35.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.592+0000 7fdf75bdb640 1 -- 172.21.15.44:0/2522294716 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf701552a0 msgr2=0x7fdf7015bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:35.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.592+0000 7fdf75bdb640 1 --2- 172.21.15.44:0/2522294716 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf701552a0 0x7fdf7015bb80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fdf600097d0 tx=0x7fdf6002b9c0 comp rx=0 tx=0).stop 2024-04-22T06:31:35.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.592+0000 7fdf75bdb640 1 -- 172.21.15.44:0/2522294716 shutdown_connections 2024-04-22T06:31:35.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.592+0000 7fdf75bdb640 1 --2- 172.21.15.44:0/2522294716 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf701552a0 0x7fdf7015bb80 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:35.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.592+0000 7fdf75bdb640 1 -- 172.21.15.44:0/2522294716 >> 172.21.15.44:0/2522294716 conn(0x7fdf70075820 msgr2=0x7fdf70073000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:35.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.592+0000 7fdf75bdb640 1 -- 172.21.15.44:0/2522294716 shutdown_connections 2024-04-22T06:31:35.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.592+0000 7fdf75bdb640 1 -- 172.21.15.44:0/2522294716 wait complete. 2024-04-22T06:31:35.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.593+0000 7fdf75bdb640 1 Processor -- start 2024-04-22T06:31:35.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.594+0000 7fdf75bdb640 1 -- start start 2024-04-22T06:31:35.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.594+0000 7fdf75bdb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf700cc4e0 0x7fdf7014dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:35.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.594+0000 7fdf75bdb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf701552a0 0x7fdf701f3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:35.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.594+0000 7fdf75bdb640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf701f4370 0x7fdf7015ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:35.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.594+0000 7fdf75bdb640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7006d540 con 0x7fdf700cc4e0 2024-04-22T06:31:35.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.594+0000 7fdf75bdb640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7006d260 con 0x7fdf701552a0 2024-04-22T06:31:35.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.594+0000 7fdf75bdb640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdf700b6690 con 0x7fdf701f4370 2024-04-22T06:31:35.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.594+0000 7fdf6f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf700cc4e0 0x7fdf7014dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:35.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.594+0000 7fdf6f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf700cc4e0 0x7fdf7014dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53434/0 (socket says 172.21.15.44:53434) 2024-04-22T06:31:35.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.594+0000 7fdf6f7fe640 1 -- 172.21.15.44:0/2667929077 learned_addr learned my addr 172.21.15.44:0/2667929077 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:35.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.594+0000 7fdf6ffff640 1 --2- 172.21.15.44:0/2667929077 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf701f4370 0x7fdf7015ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:35.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.594+0000 7fdf6effd640 1 --2- 172.21.15.44:0/2667929077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf701552a0 0x7fdf701f3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:35.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.595+0000 7fdf6f7fe640 1 -- 172.21.15.44:0/2667929077 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf701f4370 msgr2=0x7fdf7015ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:35.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.595+0000 7fdf6f7fe640 1 --2- 172.21.15.44:0/2667929077 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf701f4370 0x7fdf7015ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:35.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.595+0000 7fdf6f7fe640 1 -- 172.21.15.44:0/2667929077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf701552a0 msgr2=0x7fdf701f3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:35.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.595+0000 7fdf6f7fe640 1 --2- 172.21.15.44:0/2667929077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf701552a0 0x7fdf701f3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:35.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.595+0000 7fdf6f7fe640 1 -- 172.21.15.44:0/2667929077 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf70159d40 con 0x7fdf700cc4e0 2024-04-22T06:31:35.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.595+0000 7fdf6effd640 1 --2- 172.21.15.44:0/2667929077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf701552a0 0x7fdf701f3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:31:35.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.595+0000 7fdf6f7fe640 1 --2- 172.21.15.44:0/2667929077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf700cc4e0 0x7fdf7014dad0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdf600098f0 tx=0x7fdf600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:35.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.595+0000 7fdf57fff640 1 -- 172.21.15.44:0/2667929077 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf60048070 con 0x7fdf700cc4e0 2024-04-22T06:31:35.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.595+0000 7fdf57fff640 1 -- 172.21.15.44:0/2667929077 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf60030680 con 0x7fdf700cc4e0 2024-04-22T06:31:35.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.595+0000 7fdf57fff640 1 -- 172.21.15.44:0/2667929077 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf60034040 con 0x7fdf700cc4e0 2024-04-22T06:31:35.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.595+0000 7fdf75bdb640 1 -- 172.21.15.44:0/2667929077 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf701f4940 con 0x7fdf700cc4e0 2024-04-22T06:31:35.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.595+0000 7fdf75bdb640 1 -- 172.21.15.44:0/2667929077 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf7014e9f0 con 0x7fdf700cc4e0 2024-04-22T06:31:35.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.596+0000 7fdf75bdb640 1 -- 172.21.15.44:0/2667929077 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf70068cc0 con 0x7fdf700cc4e0 2024-04-22T06:31:35.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.596+0000 7fdf57fff640 1 -- 172.21.15.44:0/2667929077 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdf6004b070 con 0x7fdf700cc4e0 2024-04-22T06:31:35.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.597+0000 7fdf57fff640 1 --2- 172.21.15.44:0/2667929077 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fdf5c06c8f0 0x7fdf5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:35.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.597+0000 7fdf57fff640 1 -- 172.21.15.44:0/2667929077 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fdf600adb50 con 0x7fdf700cc4e0 2024-04-22T06:31:35.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.597+0000 7fdf6effd640 1 --2- 172.21.15.44:0/2667929077 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fdf5c06c8f0 0x7fdf5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:35.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.598+0000 7fdf6effd640 1 --2- 172.21.15.44:0/2667929077 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fdf5c06c8f0 0x7fdf5c06edb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdf700b63a0 tx=0x7fdf64009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:35.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.601+0000 7fdf57fff640 1 -- 172.21.15.44:0/2667929077 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdf6007b560 con 0x7fdf700cc4e0 2024-04-22T06:31:35.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.835+0000 7fdf75bdb640 1 -- 172.21.15.44:0/2667929077 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fdf700ce8a0 con 0x7fdf700cc4e0 2024-04-22T06:31:35.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.838+0000 7fdf57fff640 1 -- 172.21.15.44:0/2667929077 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174136 (secure 0 0 0) 0x7fdf6007f860 con 0x7fdf700cc4e0 2024-04-22T06:31:35.840 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:31:35.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.840+0000 7fdf75bdb640 1 -- 172.21.15.44:0/2667929077 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fdf5c06c8f0 msgr2=0x7fdf5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:35.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.840+0000 7fdf75bdb640 1 --2- 172.21.15.44:0/2667929077 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fdf5c06c8f0 0x7fdf5c06edb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdf700b63a0 tx=0x7fdf64009040 comp rx=0 tx=0).stop 2024-04-22T06:31:35.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.840+0000 7fdf75bdb640 1 -- 172.21.15.44:0/2667929077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf700cc4e0 msgr2=0x7fdf7014dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:35.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.840+0000 7fdf75bdb640 1 --2- 172.21.15.44:0/2667929077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf700cc4e0 0x7fdf7014dad0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdf600098f0 tx=0x7fdf600046c0 comp rx=0 tx=0).stop 2024-04-22T06:31:35.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.840+0000 7fdf75bdb640 1 -- 172.21.15.44:0/2667929077 shutdown_connections 2024-04-22T06:31:35.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.840+0000 7fdf75bdb640 1 --2- 172.21.15.44:0/2667929077 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fdf5c06c8f0 0x7fdf5c06edb0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:35.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.840+0000 7fdf75bdb640 1 --2- 172.21.15.44:0/2667929077 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf701f4370 0x7fdf7015ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:35.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.840+0000 7fdf75bdb640 1 --2- 172.21.15.44:0/2667929077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf701552a0 0x7fdf701f3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:35.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.840+0000 7fdf75bdb640 1 --2- 172.21.15.44:0/2667929077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf700cc4e0 0x7fdf7014dad0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:35.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.840+0000 7fdf75bdb640 1 -- 172.21.15.44:0/2667929077 >> 172.21.15.44:0/2667929077 conn(0x7fdf70075820 msgr2=0x7fdf700c85f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:35.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.840+0000 7fdf75bdb640 1 -- 172.21.15.44:0/2667929077 shutdown_connections 2024-04-22T06:31:35.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.840+0000 7fdf75bdb640 1 -- 172.21.15.44:0/2667929077 wait complete. 2024-04-22T06:31:35.851 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":5,"flags":0,"active_gid":4111,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2570466131},{"type":"v1","addr":"172.21.15.70:6833","nonce":2570466131}]},"active_addr":"172.21.15.70:6833/2570466131","active_change":"2024-04-22T06:30:46.536961+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":1943719131}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":255815653}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":3380121108}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":1806475372}]}]} 2024-04-22T06:31:35.854 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-22T06:31:35.854 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-22T06:31:35.855 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T06:31:35.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.955+0000 7f49618c1640 1 Processor -- start 2024-04-22T06:31:35.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.955+0000 7f49618c1640 1 -- start start 2024-04-22T06:31:35.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.956+0000 7f49618c1640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f495c1552a0 0x7f495c15bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:35.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.956+0000 7f49618c1640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f495c155680 0x7f495c0cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:35.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.956+0000 7f49618c1640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f495c0cc740 con 0x7f495c0cc310 2024-04-22T06:31:35.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.956+0000 7f49618c1640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f495c0cb990 con 0x7f495c155680 2024-04-22T06:31:35.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.956+0000 7f49618c1640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f495c06c6e0 con 0x7f495c1552a0 2024-04-22T06:31:35.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.956+0000 7f495a7fc640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f495c155680 0x7f495c0cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:35.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.956+0000 7f495b7fe640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f495c0cc310 0x7f495c0ccfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:49424/0 (socket says 172.21.15.44:49424) 2024-04-22T06:31:35.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.956+0000 7f495b7fe640 1 -- 172.21.15.44:0/1160714569 learned_addr learned my addr 172.21.15.44:0/1160714569 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:35.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.956+0000 7f495affd640 1 --2- 172.21.15.44:0/1160714569 >> v2:172.21.15.70:3300/0 conn(0x7f495c1552a0 0x7f495c15bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:35.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.956+0000 7f495a7fc640 1 -- 172.21.15.44:0/1160714569 >> v2:172.21.15.70:3300/0 conn(0x7f495c1552a0 msgr2=0x7f495c15bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:35.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.956+0000 7f495a7fc640 1 --2- 172.21.15.44:0/1160714569 >> v2:172.21.15.70:3300/0 conn(0x7f495c1552a0 0x7f495c15bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:35.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.956+0000 7f495a7fc640 1 -- 172.21.15.44:0/1160714569 >> v1:172.21.15.44:6789/0 conn(0x7f495c0cc310 legacy=0x7f495c0ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:35.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.956+0000 7f495a7fc640 1 -- 172.21.15.44:0/1160714569 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495c06c9c0 con 0x7f495c155680 2024-04-22T06:31:35.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.956+0000 7f4959ffb640 1 -- 172.21.15.44:0/1160714569 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3761700124 0 0) 0x7f495c0cc740 con 0x7f495c0cc310 2024-04-22T06:31:35.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.957+0000 7f495a7fc640 1 --2- 172.21.15.44:0/1160714569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f495c155680 0x7f495c0cbdd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f495c073e80 tx=0x7f495002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aac0b5d7f8de79a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:35.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.957+0000 7f4959ffb640 1 -- 172.21.15.44:0/1160714569 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4950034070 con 0x7f495c155680 2024-04-22T06:31:35.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.957+0000 7f4959ffb640 1 -- 172.21.15.44:0/1160714569 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4950030610 con 0x7f495c155680 2024-04-22T06:31:35.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.957+0000 7f4959ffb640 1 -- 172.21.15.44:0/1160714569 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49500309e0 con 0x7f495c155680 2024-04-22T06:31:35.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.958+0000 7f49618c1640 1 -- 172.21.15.44:0/1160714569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f495c155680 msgr2=0x7f495c0cbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:35.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.958+0000 7f49618c1640 1 --2- 172.21.15.44:0/1160714569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f495c155680 0x7f495c0cbdd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f495c073e80 tx=0x7f495002b960 comp rx=0 tx=0).stop 2024-04-22T06:31:35.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.958+0000 7f49618c1640 1 -- 172.21.15.44:0/1160714569 shutdown_connections 2024-04-22T06:31:35.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.958+0000 7f49618c1640 1 --2- 172.21.15.44:0/1160714569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f495c155680 0x7f495c0cbdd0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:35.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.958+0000 7f49618c1640 1 --2- 172.21.15.44:0/1160714569 >> v2:172.21.15.70:3300/0 conn(0x7f495c1552a0 0x7f495c15bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:35.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.958+0000 7f49618c1640 1 -- 172.21.15.44:0/1160714569 >> 172.21.15.44:0/1160714569 conn(0x7f495c075820 msgr2=0x7f495c073000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:35.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.958+0000 7f49618c1640 1 -- 172.21.15.44:0/1160714569 shutdown_connections 2024-04-22T06:31:35.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.958+0000 7f49618c1640 1 -- 172.21.15.44:0/1160714569 wait complete. 2024-04-22T06:31:35.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.959+0000 7f49618c1640 1 Processor -- start 2024-04-22T06:31:35.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.959+0000 7f49618c1640 1 -- start start 2024-04-22T06:31:35.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.960+0000 7f49618c1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f495c0cc310 0x7f495c1f10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:35.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.960+0000 7f49618c1640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f495c1552a0 0x7f495c0c26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:35.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.960+0000 7f49618c1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f495c1566f0 0x7f495c0c2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:35.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.960+0000 7f49618c1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f495c06d260 con 0x7f495c1566f0 2024-04-22T06:31:35.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.960+0000 7f49618c1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f495c06cf80 con 0x7f495c0cc310 2024-04-22T06:31:35.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.960+0000 7f49618c1640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f495c06d540 con 0x7f495c1552a0 2024-04-22T06:31:35.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.960+0000 7f495b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f495c1566f0 0x7f495c0c2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:35.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.960+0000 7f495b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f495c1566f0 0x7f495c0c2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53446/0 (socket says 172.21.15.44:53446) 2024-04-22T06:31:35.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.960+0000 7f495b7fe640 1 -- 172.21.15.44:0/1051723818 learned_addr learned my addr 172.21.15.44:0/1051723818 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:35.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.960+0000 7f495a7fc640 1 --2- 172.21.15.44:0/1051723818 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f495c1552a0 0x7f495c0c26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:35.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.960+0000 7f495affd640 1 --2- 172.21.15.44:0/1051723818 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f495c0cc310 0x7f495c1f10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:35.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.960+0000 7f495b7fe640 1 -- 172.21.15.44:0/1051723818 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f495c1552a0 msgr2=0x7f495c0c26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.960+0000 7f495b7fe640 1 --2- 172.21.15.44:0/1051723818 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f495c1552a0 0x7f495c0c26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.960+0000 7f495b7fe640 1 -- 172.21.15.44:0/1051723818 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f495c0cc310 msgr2=0x7f495c1f10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.960+0000 7f495b7fe640 1 --2- 172.21.15.44:0/1051723818 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f495c0cc310 0x7f495c1f10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.960+0000 7f495b7fe640 1 -- 172.21.15.44:0/1051723818 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495c1f15f0 con 0x7f495c1566f0 2024-04-22T06:31:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.960+0000 7f495b7fe640 1 --2- 172.21.15.44:0/1051723818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f495c1566f0 0x7f495c0c2c20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f495400c7d0 tx=0x7f495400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.960+0000 7f4943fff640 1 -- 172.21.15.44:0/1051723818 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4954018070 con 0x7f495c1566f0 2024-04-22T06:31:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.960+0000 7f4943fff640 1 -- 172.21.15.44:0/1051723818 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f495400a5e0 con 0x7f495c1566f0 2024-04-22T06:31:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.960+0000 7f49618c1640 1 -- 172.21.15.44:0/1051723818 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f495c158c60 con 0x7f495c1566f0 2024-04-22T06:31:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.960+0000 7f4943fff640 1 -- 172.21.15.44:0/1051723818 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495400aa10 con 0x7f495c1566f0 2024-04-22T06:31:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.961+0000 7f49618c1640 1 -- 172.21.15.44:0/1051723818 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f495c0d0cb0 con 0x7f495c1566f0 2024-04-22T06:31:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.961+0000 7f49618c1640 1 -- 172.21.15.44:0/1051723818 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f495c068cc0 con 0x7f495c1566f0 2024-04-22T06:31:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.962+0000 7f4943fff640 1 -- 172.21.15.44:0/1051723818 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f495401b070 con 0x7f495c1566f0 2024-04-22T06:31:35.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.962+0000 7f4943fff640 1 --2- 172.21.15.44:0/1051723818 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f494806c8f0 0x7f494806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:35.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.962+0000 7f4943fff640 1 -- 172.21.15.44:0/1051723818 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f495408b4e0 con 0x7f495c1566f0 2024-04-22T06:31:35.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.963+0000 7f495affd640 1 --2- 172.21.15.44:0/1051723818 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f494806c8f0 0x7f494806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:35.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.963+0000 7f495affd640 1 --2- 172.21.15.44:0/1051723818 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f494806c8f0 0x7f494806edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f494c006fd0 tx=0x7f494c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:35.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:35.967+0000 7f4943fff640 1 -- 172.21.15.44:0/1051723818 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4954058e50 con 0x7f495c1566f0 2024-04-22T06:31:36.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.153+0000 7f49618c1640 1 -- 172.21.15.44:0/1051723818 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f495c154760 con 0x7f495c1566f0 2024-04-22T06:31:36.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.154+0000 7f4943fff640 1 -- 172.21.15.44:0/1051723818 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18466 (secure 0 0 0) 0x7f495405d150 con 0x7f495c1566f0 2024-04-22T06:31:36.154 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:31:36.155 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"fsid":"9b0a34ca-403d-4513-8a66-d3749ba559aa","created":"2024-04-22T06:30:42.334581+0000","modified":"2024-04-22T06:31:11.406256+0000","last_up_change":"2024-04-22T06:30:52.942257+0000","last_in_change":"2024-04-22T06:30:44.467727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T06:30:54.560256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T06:31:04.612387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T06:31:05.312124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-22T06:31:09.025550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b4a6692-53c9-472d-b2f3-1dd29a6679ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6817","nonce":2220268493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6819","nonce":2220268493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6823","nonce":2220268493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6821","nonce":2220268493}]},"public_addr":"172.21.15.44:6817/2220268493","cluster_addr":"172.21.15.44:6819/2220268493","heartbeat_back_addr":"172.21.15.44:6823/2220268493","heartbeat_front_addr":"172.21.15.44:6821/2220268493","state":["exists","up"]},{"osd":1,"uuid":"70a6370a-7c0a-4d64-a0e4-b2e726b4f291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6809","nonce":1278678977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6811","nonce":1278678977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6815","nonce":1278678977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6813","nonce":1278678977}]},"public_addr":"172.21.15.44:6809/1278678977","cluster_addr":"172.21.15.44:6811/1278678977","heartbeat_back_addr":"172.21.15.44:6815/1278678977","heartbeat_front_addr":"172.21.15.44:6813/1278678977","state":["exists","up"]},{"osd":2,"uuid":"f2e51a62-ca91-470d-bd5a-5710c61deee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6801","nonce":2544087903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6803","nonce":2544087903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6807","nonce":2544087903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6805","nonce":2544087903}]},"public_addr":"172.21.15.44:6801/2544087903","cluster_addr":"172.21.15.44:6803/2544087903","heartbeat_back_addr":"172.21.15.44:6807/2544087903","heartbeat_front_addr":"172.21.15.44:6805/2544087903","state":["exists","up"]},{"osd":3,"uuid":"bc514203-9af9-4f1f-b18e-765bdd5de6e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6825","nonce":1692928109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6827","nonce":1692928109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6831","nonce":1692928109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6829","nonce":1692928109}]},"public_addr":"172.21.15.44:6825/1692928109","cluster_addr":"172.21.15.44:6827/1692928109","heartbeat_back_addr":"172.21.15.44:6831/1692928109","heartbeat_front_addr":"172.21.15.44:6829/1692928109","state":["exists","up"]},{"osd":4,"uuid":"357c439a-6e08-4c19-ac18-b68e6c890497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6820","nonce":3802577427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6824","nonce":3802577427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6831","nonce":3802577427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6828","nonce":3802577427}]},"public_addr":"172.21.15.70:6820/3802577427","cluster_addr":"172.21.15.70:6824/3802577427","heartbeat_back_addr":"172.21.15.70:6831/3802577427","heartbeat_front_addr":"172.21.15.70:6828/3802577427","state":["exists","up"]},{"osd":5,"uuid":"952fb88f-d56e-4305-bfa8-d32674332dfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6801","nonce":664757345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6803","nonce":664757345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6807","nonce":664757345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6805","nonce":664757345}]},"public_addr":"172.21.15.70:6801/664757345","cluster_addr":"172.21.15.70:6803/664757345","heartbeat_back_addr":"172.21.15.70:6807/664757345","heartbeat_front_addr":"172.21.15.70:6805/664757345","state":["exists","up"]},{"osd":6,"uuid":"74399b3d-fac2-45ab-9172-2cd95b02450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6817","nonce":1623253482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6821","nonce":1623253482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6827","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6829","nonce":1623253482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6825","nonce":1623253482}]},"public_addr":"172.21.15.70:6817/1623253482","cluster_addr":"172.21.15.70:6821/1623253482","heartbeat_back_addr":"172.21.15.70:6829/1623253482","heartbeat_front_addr":"172.21.15.70:6825/1623253482","state":["exists","up"]},{"osd":7,"uuid":"8f35508d-6fcc-43ff-8ee7-0498a7ab3b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6809","nonce":1551180300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6811","nonce":1551180300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6815","nonce":1551180300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6813","nonce":1551180300}]},"public_addr":"172.21.15.70:6809/1551180300","cluster_addr":"172.21.15.70:6811/1551180300","heartbeat_back_addr":"172.21.15.70:6815/1551180300","heartbeat_front_addr":"172.21.15.70:6813/1551180300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.612502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.653756+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.521169+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.649955+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.640231+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.686638+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.642199+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.700360+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T06:31:36.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.156+0000 7f49618c1640 1 -- 172.21.15.44:0/1051723818 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f494806c8f0 msgr2=0x7f494806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.156+0000 7f49618c1640 1 --2- 172.21.15.44:0/1051723818 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f494806c8f0 0x7f494806edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f494c006fd0 tx=0x7f494c008040 comp rx=0 tx=0).stop 2024-04-22T06:31:36.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.156+0000 7f49618c1640 1 -- 172.21.15.44:0/1051723818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f495c1566f0 msgr2=0x7f495c0c2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.156+0000 7f49618c1640 1 --2- 172.21.15.44:0/1051723818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f495c1566f0 0x7f495c0c2c20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f495400c7d0 tx=0x7f495400cc90 comp rx=0 tx=0).stop 2024-04-22T06:31:36.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.156+0000 7f49618c1640 1 -- 172.21.15.44:0/1051723818 shutdown_connections 2024-04-22T06:31:36.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.156+0000 7f49618c1640 1 --2- 172.21.15.44:0/1051723818 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f494806c8f0 0x7f494806edb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.156+0000 7f49618c1640 1 --2- 172.21.15.44:0/1051723818 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f495c1552a0 0x7f495c0c26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.156+0000 7f49618c1640 1 --2- 172.21.15.44:0/1051723818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f495c1566f0 0x7f495c0c2c20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.156+0000 7f49618c1640 1 --2- 172.21.15.44:0/1051723818 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f495c0cc310 0x7f495c1f10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.156+0000 7f49618c1640 1 -- 172.21.15.44:0/1051723818 >> 172.21.15.44:0/1051723818 conn(0x7f495c075820 msgr2=0x7f495c0c9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:36.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.157+0000 7f49618c1640 1 -- 172.21.15.44:0/1051723818 shutdown_connections 2024-04-22T06:31:36.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.157+0000 7f49618c1640 1 -- 172.21.15.44:0/1051723818 wait complete. 2024-04-22T06:31:36.167 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-22T06:31:36.168 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T06:31:36.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.269+0000 7fb2959f4640 1 Processor -- start 2024-04-22T06:31:36.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.269+0000 7fb2959f4640 1 -- start start 2024-04-22T06:31:36.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.270+0000 7fb2959f4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb29015e610 0x7fb2900cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.270+0000 7fb2959f4640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb2900cc310 0x7fb2900cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.270+0000 7fb2959f4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2900c5b70 con 0x7fb290155340 2024-04-22T06:31:36.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.270+0000 7fb2959f4640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fb29015b020 con 0x7fb2900cc310 2024-04-22T06:31:36.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.270+0000 7fb2959f4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fb290155720 con 0x7fb29015e610 2024-04-22T06:31:36.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.270+0000 7fb28f7fe640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb2900cc310 0x7fb2900cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.270+0000 7fb28f7fe640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb2900cc310 0x7fb2900cb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53454/0 (socket says 172.21.15.44:53454) 2024-04-22T06:31:36.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.270+0000 7fb28f7fe640 1 -- 172.21.15.44:0/4086427258 learned_addr learned my addr 172.21.15.44:0/4086427258 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:36.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.270+0000 7fb287fff640 1 --2- 172.21.15.44:0/4086427258 >> v2:172.21.15.70:3300/0 conn(0x7fb29015e610 0x7fb2900cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.270+0000 7fb28f7fe640 1 -- 172.21.15.44:0/4086427258 >> v2:172.21.15.70:3300/0 conn(0x7fb29015e610 msgr2=0x7fb2900cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:36.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.270+0000 7fb28f7fe640 1 --2- 172.21.15.44:0/4086427258 >> v2:172.21.15.70:3300/0 conn(0x7fb29015e610 0x7fb2900cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.270+0000 7fb28f7fe640 1 -- 172.21.15.44:0/4086427258 >> v1:172.21.15.70:6789/0 conn(0x7fb290155340 legacy=0x7fb29015b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.270+0000 7fb28f7fe640 1 -- 172.21.15.44:0/4086427258 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb29015eb50 con 0x7fb2900cc310 2024-04-22T06:31:36.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.271+0000 7fb28f7fe640 1 --2- 172.21.15.44:0/4086427258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2900cc310 0x7fb2900cb500 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb2900cde40 tx=0x7fb28802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e40ba4b8e0abc2e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:36.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.271+0000 7fb28e7fc640 1 -- 172.21.15.44:0/4086427258 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28802cb20 con 0x7fb2900cc310 2024-04-22T06:31:36.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.271+0000 7fb28e7fc640 1 -- 172.21.15.44:0/4086427258 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb28802ccc0 con 0x7fb2900cc310 2024-04-22T06:31:36.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.271+0000 7fb28e7fc640 1 -- 172.21.15.44:0/4086427258 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288034610 con 0x7fb2900cc310 2024-04-22T06:31:36.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.272+0000 7fb2959f4640 1 -- 172.21.15.44:0/4086427258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2900cc310 msgr2=0x7fb2900cb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.272+0000 7fb2959f4640 1 --2- 172.21.15.44:0/4086427258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2900cc310 0x7fb2900cb500 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb2900cde40 tx=0x7fb28802bf80 comp rx=0 tx=0).stop 2024-04-22T06:31:36.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.272+0000 7fb2959f4640 1 -- 172.21.15.44:0/4086427258 shutdown_connections 2024-04-22T06:31:36.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.272+0000 7fb2959f4640 1 --2- 172.21.15.44:0/4086427258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2900cc310 0x7fb2900cb500 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.272+0000 7fb2959f4640 1 --2- 172.21.15.44:0/4086427258 >> v2:172.21.15.70:3300/0 conn(0x7fb29015e610 0x7fb2900cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.272+0000 7fb2959f4640 1 -- 172.21.15.44:0/4086427258 >> 172.21.15.44:0/4086427258 conn(0x7fb29006c560 msgr2=0x7fb290074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:36.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.272+0000 7fb2959f4640 1 -- 172.21.15.44:0/4086427258 shutdown_connections 2024-04-22T06:31:36.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.272+0000 7fb2959f4640 1 -- 172.21.15.44:0/4086427258 wait complete. 2024-04-22T06:31:36.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.273+0000 7fb2959f4640 1 Processor -- start 2024-04-22T06:31:36.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.273+0000 7fb2959f4640 1 -- start start 2024-04-22T06:31:36.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.274+0000 7fb2959f4640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb2900cc310 0x7fb2900b48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.274+0000 7fb2959f4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb290155340 0x7fb2901491a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.274+0000 7fb2959f4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb29015e610 0x7fb2900ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.274+0000 7fb2959f4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb29015f3f0 con 0x7fb29015e610 2024-04-22T06:31:36.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.274+0000 7fb2959f4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb29015f110 con 0x7fb290155340 2024-04-22T06:31:36.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.274+0000 7fb2959f4640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb29015b2e0 con 0x7fb2900cc310 2024-04-22T06:31:36.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.274+0000 7fb28f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb29015e610 0x7fb2900ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.274+0000 7fb28f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb29015e610 0x7fb2900ca1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53458/0 (socket says 172.21.15.44:53458) 2024-04-22T06:31:36.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.274+0000 7fb28f7fe640 1 -- 172.21.15.44:0/1231745811 learned_addr learned my addr 172.21.15.44:0/1231745811 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:36.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.274+0000 7fb28effd640 1 --2- 172.21.15.44:0/1231745811 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb2900cc310 0x7fb2900b48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.274+0000 7fb287fff640 1 --2- 172.21.15.44:0/1231745811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb290155340 0x7fb2901491a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.274+0000 7fb28f7fe640 1 -- 172.21.15.44:0/1231745811 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb2900cc310 msgr2=0x7fb2900b48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.274+0000 7fb28f7fe640 1 --2- 172.21.15.44:0/1231745811 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb2900cc310 0x7fb2900b48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.274+0000 7fb28f7fe640 1 -- 172.21.15.44:0/1231745811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb290155340 msgr2=0x7fb2901491a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.274+0000 7fb28f7fe640 1 --2- 172.21.15.44:0/1231745811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb290155340 0x7fb2901491a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.274+0000 7fb28f7fe640 1 -- 172.21.15.44:0/1231745811 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2901f2020 con 0x7fb29015e610 2024-04-22T06:31:36.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.274+0000 7fb28f7fe640 1 --2- 172.21.15.44:0/1231745811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb29015e610 0x7fb2900ca1f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb28802c590 tx=0x7fb288004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:36.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.274+0000 7fb28cff9640 1 -- 172.21.15.44:0/1231745811 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288043020 con 0x7fb29015e610 2024-04-22T06:31:36.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.275+0000 7fb28cff9640 1 -- 172.21.15.44:0/1231745811 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb28803e450 con 0x7fb29015e610 2024-04-22T06:31:36.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.275+0000 7fb2959f4640 1 -- 172.21.15.44:0/1231745811 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2901f1d40 con 0x7fb29015e610 2024-04-22T06:31:36.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.275+0000 7fb28cff9640 1 -- 172.21.15.44:0/1231745811 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288009510 con 0x7fb29015e610 2024-04-22T06:31:36.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.275+0000 7fb2959f4640 1 -- 172.21.15.44:0/1231745811 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb29014f960 con 0x7fb29015e610 2024-04-22T06:31:36.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.275+0000 7fb2959f4640 1 -- 172.21.15.44:0/1231745811 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb290075f80 con 0x7fb29015e610 2024-04-22T06:31:36.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.276+0000 7fb28cff9640 1 -- 172.21.15.44:0/1231745811 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb288046020 con 0x7fb29015e610 2024-04-22T06:31:36.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.276+0000 7fb28cff9640 1 --2- 172.21.15.44:0/1231745811 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb27006c8f0 0x7fb27006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.277+0000 7fb28cff9640 1 -- 172.21.15.44:0/1231745811 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb2880ae830 con 0x7fb29015e610 2024-04-22T06:31:36.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.277+0000 7fb28effd640 1 --2- 172.21.15.44:0/1231745811 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb27006c8f0 0x7fb27006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.277+0000 7fb28effd640 1 --2- 172.21.15.44:0/1231745811 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb27006c8f0 0x7fb27006edb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb280001780 tx=0x7fb280009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:36.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.281+0000 7fb28cff9640 1 -- 172.21.15.44:0/1231745811 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb290075f80 con 0x7fb29015e610 2024-04-22T06:31:36.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.469+0000 7fb2959f4640 1 -- 172.21.15.44:0/1231745811 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb2900caf60 con 0x7fb29015e610 2024-04-22T06:31:36.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.471+0000 7fb28cff9640 1 -- 172.21.15.44:0/1231745811 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18466 (secure 0 0 0) 0x7fb2900caf60 con 0x7fb29015e610 2024-04-22T06:31:36.471 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:31:36.471 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"fsid":"9b0a34ca-403d-4513-8a66-d3749ba559aa","created":"2024-04-22T06:30:42.334581+0000","modified":"2024-04-22T06:31:11.406256+0000","last_up_change":"2024-04-22T06:30:52.942257+0000","last_in_change":"2024-04-22T06:30:44.467727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T06:30:54.560256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T06:31:04.612387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T06:31:05.312124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-22T06:31:09.025550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b4a6692-53c9-472d-b2f3-1dd29a6679ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6817","nonce":2220268493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6819","nonce":2220268493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6823","nonce":2220268493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6821","nonce":2220268493}]},"public_addr":"172.21.15.44:6817/2220268493","cluster_addr":"172.21.15.44:6819/2220268493","heartbeat_back_addr":"172.21.15.44:6823/2220268493","heartbeat_front_addr":"172.21.15.44:6821/2220268493","state":["exists","up"]},{"osd":1,"uuid":"70a6370a-7c0a-4d64-a0e4-b2e726b4f291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6809","nonce":1278678977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6811","nonce":1278678977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6815","nonce":1278678977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6813","nonce":1278678977}]},"public_addr":"172.21.15.44:6809/1278678977","cluster_addr":"172.21.15.44:6811/1278678977","heartbeat_back_addr":"172.21.15.44:6815/1278678977","heartbeat_front_addr":"172.21.15.44:6813/1278678977","state":["exists","up"]},{"osd":2,"uuid":"f2e51a62-ca91-470d-bd5a-5710c61deee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6801","nonce":2544087903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6803","nonce":2544087903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6807","nonce":2544087903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6805","nonce":2544087903}]},"public_addr":"172.21.15.44:6801/2544087903","cluster_addr":"172.21.15.44:6803/2544087903","heartbeat_back_addr":"172.21.15.44:6807/2544087903","heartbeat_front_addr":"172.21.15.44:6805/2544087903","state":["exists","up"]},{"osd":3,"uuid":"bc514203-9af9-4f1f-b18e-765bdd5de6e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6825","nonce":1692928109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6827","nonce":1692928109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6831","nonce":1692928109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6829","nonce":1692928109}]},"public_addr":"172.21.15.44:6825/1692928109","cluster_addr":"172.21.15.44:6827/1692928109","heartbeat_back_addr":"172.21.15.44:6831/1692928109","heartbeat_front_addr":"172.21.15.44:6829/1692928109","state":["exists","up"]},{"osd":4,"uuid":"357c439a-6e08-4c19-ac18-b68e6c890497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6820","nonce":3802577427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6824","nonce":3802577427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6831","nonce":3802577427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6828","nonce":3802577427}]},"public_addr":"172.21.15.70:6820/3802577427","cluster_addr":"172.21.15.70:6824/3802577427","heartbeat_back_addr":"172.21.15.70:6831/3802577427","heartbeat_front_addr":"172.21.15.70:6828/3802577427","state":["exists","up"]},{"osd":5,"uuid":"952fb88f-d56e-4305-bfa8-d32674332dfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6801","nonce":664757345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6803","nonce":664757345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6807","nonce":664757345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6805","nonce":664757345}]},"public_addr":"172.21.15.70:6801/664757345","cluster_addr":"172.21.15.70:6803/664757345","heartbeat_back_addr":"172.21.15.70:6807/664757345","heartbeat_front_addr":"172.21.15.70:6805/664757345","state":["exists","up"]},{"osd":6,"uuid":"74399b3d-fac2-45ab-9172-2cd95b02450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6817","nonce":1623253482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6821","nonce":1623253482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6827","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6829","nonce":1623253482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6825","nonce":1623253482}]},"public_addr":"172.21.15.70:6817/1623253482","cluster_addr":"172.21.15.70:6821/1623253482","heartbeat_back_addr":"172.21.15.70:6829/1623253482","heartbeat_front_addr":"172.21.15.70:6825/1623253482","state":["exists","up"]},{"osd":7,"uuid":"8f35508d-6fcc-43ff-8ee7-0498a7ab3b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6809","nonce":1551180300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6811","nonce":1551180300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6815","nonce":1551180300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6813","nonce":1551180300}]},"public_addr":"172.21.15.70:6809/1551180300","cluster_addr":"172.21.15.70:6811/1551180300","heartbeat_back_addr":"172.21.15.70:6815/1551180300","heartbeat_front_addr":"172.21.15.70:6813/1551180300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.612502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.653756+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.521169+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.649955+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.640231+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.686638+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.642199+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.700360+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T06:31:36.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.473+0000 7fb2959f4640 1 -- 172.21.15.44:0/1231745811 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb27006c8f0 msgr2=0x7fb27006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.473+0000 7fb2959f4640 1 --2- 172.21.15.44:0/1231745811 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb27006c8f0 0x7fb27006edb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb280001780 tx=0x7fb280009040 comp rx=0 tx=0).stop 2024-04-22T06:31:36.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.473+0000 7fb2959f4640 1 -- 172.21.15.44:0/1231745811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb29015e610 msgr2=0x7fb2900ca1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.473+0000 7fb2959f4640 1 --2- 172.21.15.44:0/1231745811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb29015e610 0x7fb2900ca1f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb28802c590 tx=0x7fb288004a40 comp rx=0 tx=0).stop 2024-04-22T06:31:36.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.473+0000 7fb2959f4640 1 -- 172.21.15.44:0/1231745811 shutdown_connections 2024-04-22T06:31:36.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.473+0000 7fb2959f4640 1 --2- 172.21.15.44:0/1231745811 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb27006c8f0 0x7fb27006edb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.473+0000 7fb2959f4640 1 --2- 172.21.15.44:0/1231745811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb29015e610 0x7fb2900ca1f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.473+0000 7fb2959f4640 1 --2- 172.21.15.44:0/1231745811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb290155340 0x7fb2901491a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.473+0000 7fb2959f4640 1 --2- 172.21.15.44:0/1231745811 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb2900cc310 0x7fb2900b48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.473+0000 7fb2959f4640 1 -- 172.21.15.44:0/1231745811 >> 172.21.15.44:0/1231745811 conn(0x7fb29006c560 msgr2=0x7fb29015b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:36.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.473+0000 7fb2959f4640 1 -- 172.21.15.44:0/1231745811 shutdown_connections 2024-04-22T06:31:36.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.473+0000 7fb2959f4640 1 -- 172.21.15.44:0/1231745811 wait complete. 2024-04-22T06:31:36.484 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-22T06:31:36.484 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-22T06:31:36.485 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-22T06:31:36.485 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-22T06:31:36.486 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-22T06:31:36.486 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-22T06:31:36.486 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-22T06:31:36.487 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-22T06:31:36.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.660+0000 7f8adf674640 1 Processor -- start 2024-04-22T06:31:36.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.660+0000 7fc924aa8640 1 Processor -- start 2024-04-22T06:31:36.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.660+0000 7f3823e9a640 1 Processor -- start 2024-04-22T06:31:36.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.660+0000 7fc924aa8640 1 -- start start 2024-04-22T06:31:36.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.662+0000 7fc924aa8640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc92015f530 0x7fc9200753c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.662+0000 7fc924aa8640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc920162ca0 con 0x7fc9201595b0 2024-04-22T06:31:36.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.662+0000 7fc924aa8640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc920162f60 con 0x7fc920070300 2024-04-22T06:31:36.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.662+0000 7fc924aa8640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fc920163240 con 0x7fc92015f530 2024-04-22T06:31:36.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.660+0000 7f8adf674640 1 -- start start 2024-04-22T06:31:36.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.660+0000 7f3823e9a640 1 -- start start 2024-04-22T06:31:36.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.662+0000 7f3823e9a640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f38140dd4f0 0x7f3814050fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.661+0000 7f8adf674640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ad8070300 0x7f8ad815f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.661+0000 7f8adf674640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f8ad8074cb0 0x7f8ad8075140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.661+0000 7f8adf674640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ad8162ca0 con 0x7f8ad81595b0 2024-04-22T06:31:36.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.661+0000 7f8adf674640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f8ad8162f60 con 0x7f8ad8074cb0 2024-04-22T06:31:36.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.661+0000 7f8adf674640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8ad8163240 con 0x7f8ad8070300 2024-04-22T06:31:36.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.662+0000 7fc91e59c640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fc9201595b0 0x7fc92006fbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:49428/0 (socket says 172.21.15.44:49428) 2024-04-22T06:31:36.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.662+0000 7fc91e59c640 1 -- 172.21.15.44:0/125025029 learned_addr learned my addr 172.21.15.44:0/125025029 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:36.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.662+0000 7f3823e9a640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38140e6f70 0x7f38140527f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.662+0000 7f09f9873640 1 Processor -- start 2024-04-22T06:31:36.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.662+0000 7f3823e9a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38140e3610 con 0x7f3814052380 2024-04-22T06:31:36.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.662+0000 7f3823e9a640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f38140e33c0 con 0x7f38140dd4f0 2024-04-22T06:31:36.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.662+0000 7fc91ed9d640 1 --2- 172.21.15.44:0/125025029 >> v2:172.21.15.70:3300/0 conn(0x7fc92015f530 0x7fc9200753c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.662+0000 7f09f9873640 1 -- start start 2024-04-22T06:31:36.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.662+0000 7f8adcc0f640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ad8070300 0x7f8ad815f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.662+0000 7f8adcc0f640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ad8070300 0x7f8ad815f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.44:45404/0 (socket says 172.21.15.44:45404) 2024-04-22T06:31:36.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.662+0000 7f8adcc0f640 1 -- 172.21.15.44:0/1146833671 learned_addr learned my addr 172.21.15.44:0/1146833671 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:36.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.662+0000 7fc91d59a640 1 -- 172.21.15.44:0/125025029 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1850829293 0 0) 0x7fc920162ca0 con 0x7fc9201595b0 2024-04-22T06:31:36.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.663+0000 7fc91d59a640 1 -- 172.21.15.44:0/125025029 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc908003520 con 0x7fc9201595b0 2024-04-22T06:31:36.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.663+0000 7fc91d59a640 1 -- 172.21.15.44:0/125025029 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3481493559 0 0) 0x7fc920162f60 con 0x7fc920070300 2024-04-22T06:31:36.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.663+0000 7fc91d59a640 1 -- 172.21.15.44:0/125025029 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc920162ca0 con 0x7fc920070300 2024-04-22T06:31:36.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.662+0000 7f3823e9a640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f38140e3f40 con 0x7f38140e6f70 2024-04-22T06:31:36.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.662+0000 7f3822437640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38140e6f70 0x7f38140527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.662+0000 7f3822437640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38140e6f70 0x7f38140527f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.44:45408/0 (socket says 172.21.15.44:45408) 2024-04-22T06:31:36.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.662+0000 7f3822437640 1 -- 172.21.15.44:0/1057060670 learned_addr learned my addr 172.21.15.44:0/1057060670 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:36.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.663+0000 7f8adcc0f640 1 -- 172.21.15.44:0/1146833671 >> v2:172.21.15.44:3300/0 conn(0x7f8ad8074cb0 msgr2=0x7f8ad8075140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:36.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.663+0000 7f3822437640 1 -- 172.21.15.44:0/1057060670 >> v2:172.21.15.44:3300/0 conn(0x7f38140dd4f0 msgr2=0x7f3814050fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:36.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.663+0000 7f3822437640 1 --2- 172.21.15.44:0/1057060670 >> v2:172.21.15.44:3300/0 conn(0x7f38140dd4f0 0x7f3814050fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.663+0000 7f3822437640 1 -- 172.21.15.44:0/1057060670 >> v1:172.21.15.70:6789/0 conn(0x7f3814052380 legacy=0x7f38140e6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.663+0000 7f3822437640 1 -- 172.21.15.44:0/1057060670 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38140e38f0 con 0x7f38140e6f70 2024-04-22T06:31:36.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.663+0000 7f09f9873640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f09ec050ca0 0x7f09ec0dac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.663+0000 7f09f9873640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f09ec04b500 0x7f09ec04b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.663+0000 7f09f9873640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09ec007760 con 0x7f09ec005ee0 2024-04-22T06:31:36.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.663+0000 7f09f9873640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f09ec006900 con 0x7f09ec050ca0 2024-04-22T06:31:36.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.664+0000 7f09f9873640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f09ec006be0 con 0x7f09ec04b500 2024-04-22T06:31:36.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.664+0000 7f09f3fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f09ec04b500 0x7f09ec04b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.664+0000 7f09f3fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f09ec04b500 0x7f09ec04b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:34044/0 (socket says 172.21.15.44:34044) 2024-04-22T06:31:36.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.664+0000 7f09f3fff640 1 -- 172.21.15.44:0/3559714398 learned_addr learned my addr 172.21.15.44:0/3559714398 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:36.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.663+0000 7fc91ed9d640 1 -- 172.21.15.44:0/125025029 >> v1:172.21.15.70:6789/0 conn(0x7fc920070300 legacy=0x7fc920074cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.663+0000 7fc91ed9d640 1 -- 172.21.15.44:0/125025029 >> v1:172.21.15.44:6789/0 conn(0x7fc9201595b0 legacy=0x7fc92006fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.663+0000 7fc91ed9d640 1 -- 172.21.15.44:0/125025029 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc920163520 con 0x7fc92015f530 2024-04-22T06:31:36.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.664+0000 7fcecee5c640 1 Processor -- start 2024-04-22T06:31:36.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.664+0000 7f09f8871640 1 --2- 172.21.15.44:0/3559714398 >> v2:172.21.15.44:3300/0 conn(0x7f09ec050ca0 0x7f09ec0dac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.664+0000 7fcecee5c640 1 -- start start 2024-04-22T06:31:36.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.664+0000 7f09f3fff640 1 -- 172.21.15.44:0/3559714398 >> v2:172.21.15.44:3300/0 conn(0x7f09ec050ca0 msgr2=0x7f09ec0dac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:36.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.665+0000 7f09f3fff640 1 --2- 172.21.15.44:0/3559714398 >> v2:172.21.15.44:3300/0 conn(0x7f09ec050ca0 0x7f09ec0dac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.665+0000 7f09f3fff640 1 -- 172.21.15.44:0/3559714398 >> v1:172.21.15.44:6789/0 conn(0x7f09ec005ee0 legacy=0x7f09ec034a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:36.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.665+0000 7f3822437640 1 --2- 172.21.15.44:0/1057060670 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38140e6f70 0x7f38140527f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f381c065320 tx=0x7f381c0a7f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ac70c0eb8b69d76 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:36.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.665+0000 7f09f3fff640 1 -- 172.21.15.44:0/3559714398 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09ec006ec0 con 0x7f09ec04b500 2024-04-22T06:31:36.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.665+0000 7f09f8871640 1 --2- 172.21.15.44:0/3559714398 >> v2:172.21.15.44:3300/0 conn(0x7f09ec050ca0 0x7f09ec0dac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:31:36.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.665+0000 7fa1b4dbf640 1 Processor -- start 2024-04-22T06:31:36.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.665+0000 7f8adcc0f640 1 --2- 172.21.15.44:0/1146833671 >> v2:172.21.15.44:3300/0 conn(0x7f8ad8074cb0 0x7f8ad8075140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.665+0000 7f8adcc0f640 1 -- 172.21.15.44:0/1146833671 >> v1:172.21.15.44:6789/0 conn(0x7f8ad81595b0 legacy=0x7f8ad806fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:36.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.665+0000 7f8adcc0f640 1 -- 172.21.15.44:0/1146833671 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad8163520 con 0x7f8ad8070300 2024-04-22T06:31:36.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.665+0000 7fc91ed9d640 1 --2- 172.21.15.44:0/125025029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc92015f530 0x7fc9200753c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc914009720 tx=0x7fc91402b680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18eb6726a9d5217d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:36.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.665+0000 7f3820c34640 1 -- 172.21.15.44:0/1057060670 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381c0b5070 con 0x7f38140e6f70 2024-04-22T06:31:36.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.665+0000 7f3820c34640 1 -- 172.21.15.44:0/1057060670 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f381c0a8e30 con 0x7f38140e6f70 2024-04-22T06:31:36.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.665+0000 7f3820c34640 1 -- 172.21.15.44:0/1057060670 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381c0b0640 con 0x7f38140e6f70 2024-04-22T06:31:36.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.665+0000 7f314d9ef640 1 Processor -- start 2024-04-22T06:31:36.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.665+0000 7f22ccd85640 1 Processor -- start 2024-04-22T06:31:36.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.665+0000 7f22ccd85640 1 -- start start 2024-04-22T06:31:36.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.665+0000 7fc91d59a640 1 -- 172.21.15.44:0/125025029 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc914037070 con 0x7fc92015f530 2024-04-22T06:31:36.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.665+0000 7fc91d59a640 1 -- 172.21.15.44:0/125025029 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc914030510 con 0x7fc92015f530 2024-04-22T06:31:36.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.666+0000 7fcecee5c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcec8155010 0x7fcec815bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.666+0000 7f09f3fff640 1 --2- 172.21.15.44:0/3559714398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09ec04b500 0x7f09ec04b920 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f09ec0d5520 tx=0x7f09e002b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c92a2312b6243b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:36.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.666+0000 7fcecee5c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fcec806d5a0 0x7fcec80cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.666+0000 7fcecee5c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcec80d0fa0 con 0x7fcec81553f0 2024-04-22T06:31:36.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.666+0000 7fcecee5c640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fcec80d1280 con 0x7fcec806d5a0 2024-04-22T06:31:36.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.666+0000 7fcecee5c640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fcec80d1560 con 0x7fcec8155010 2024-04-22T06:31:36.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.666+0000 7f3823e9a640 1 -- 172.21.15.44:0/1057060670 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38140e6f70 msgr2=0x7f38140527f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.666+0000 7f3823e9a640 1 --2- 172.21.15.44:0/1057060670 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38140e6f70 0x7f38140527f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f381c065320 tx=0x7f381c0a7f80 comp rx=0 tx=0).stop 2024-04-22T06:31:36.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.665+0000 7f314d9ef640 1 -- start start 2024-04-22T06:31:36.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.665+0000 7fa1b4dbf640 1 -- start start 2024-04-22T06:31:36.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.666+0000 7fa1b4dbf640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fa1b0151110 0x7fa1b0157990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.666+0000 7fa1b4dbf640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa1b00c3d40 0x7fa1b00c8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.666+0000 7f3823e9a640 1 -- 172.21.15.44:0/1057060670 shutdown_connections 2024-04-22T06:31:36.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.666+0000 7f3823e9a640 1 --2- 172.21.15.44:0/1057060670 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38140e6f70 0x7f38140527f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.666+0000 7f3823e9a640 1 --2- 172.21.15.44:0/1057060670 >> v2:172.21.15.44:3300/0 conn(0x7f38140dd4f0 0x7f3814050fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.666+0000 7f3823e9a640 1 -- 172.21.15.44:0/1057060670 >> 172.21.15.44:0/1057060670 conn(0x7f381400c3a0 msgr2=0x7f381400c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:36.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.666+0000 7fceccbf8640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcec8155010 0x7fcec815bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.666+0000 7fcec7fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fcec81553f0 0x7fcec80c6550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.44:46186/0 (socket says 172.21.15.44:46186) 2024-04-22T06:31:36.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.666+0000 7fcec7fff640 1 -- 172.21.15.44:0/3991715561 learned_addr learned my addr 172.21.15.44:0/3991715561 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:36.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.666+0000 7fc91d59a640 1 -- 172.21.15.44:0/125025029 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9100015a0 con 0x7fc92015f530 2024-04-22T06:31:36.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.666+0000 7fc924aa8640 1 -- 172.21.15.44:0/125025029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc92015f530 msgr2=0x7fc9200753c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.666+0000 7fc924aa8640 1 --2- 172.21.15.44:0/125025029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc92015f530 0x7fc9200753c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc914009720 tx=0x7fc91402b680 comp rx=0 tx=0).stop 2024-04-22T06:31:36.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.666+0000 7f314d9ef640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f31400dd3f0 0x7f3140046490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.666+0000 7f22ccd85640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f22c8154f00 0x7f22c80cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7f22c659c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f22c8154f00 0x7f22c80cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7f3823e9a640 1 -- 172.21.15.44:0/1057060670 shutdown_connections 2024-04-22T06:31:36.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7fc924aa8640 1 -- 172.21.15.44:0/125025029 shutdown_connections 2024-04-22T06:31:36.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7fc924aa8640 1 --2- 172.21.15.44:0/125025029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc92015f530 0x7fc9200753c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7fc924aa8640 1 -- 172.21.15.44:0/125025029 >> 172.21.15.44:0/125025029 conn(0x7fc9200cd630 msgr2=0x7fc9200cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:36.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7f22c659c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f22c8154f00 0x7f22c80cb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53514/0 (socket says 172.21.15.44:53514) 2024-04-22T06:31:36.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7fa1af7fe640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fa1b0151110 0x7fa1b0157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7fa1af7fe640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fa1b0151110 0x7fa1b0157990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53530/0 (socket says 172.21.15.44:53530) 2024-04-22T06:31:36.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7fcec77fe640 1 -- 172.21.15.44:0/3991715561 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1000287179 0 0) 0x7fcec80d0fa0 con 0x7fcec81553f0 2024-04-22T06:31:36.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7fcec77fe640 1 -- 172.21.15.44:0/3991715561 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fceb0003520 con 0x7fcec81553f0 2024-04-22T06:31:36.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7f22c659c640 1 -- 172.21.15.44:0/2612409532 learned_addr learned my addr 172.21.15.44:0/2612409532 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:36.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7fc924aa8640 1 -- 172.21.15.44:0/125025029 shutdown_connections 2024-04-22T06:31:36.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7fcecd3f9640 1 --2- 172.21.15.44:0/3991715561 >> v2:172.21.15.44:3300/0 conn(0x7fcec806d5a0 0x7fcec80cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7f22ccd85640 1 --2- 172.21.15.44:0/2612409532 >> v2:172.21.15.70:3300/0 conn(0x7f22c806d470 0x7f22c80d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7fa1af7fe640 1 -- 172.21.15.44:0/2946745031 learned_addr learned my addr 172.21.15.44:0/2946745031 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:36.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7fa1affff640 1 --2- 172.21.15.44:0/2946745031 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa1b00c3d40 0x7fa1b00c8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7fcec77fe640 1 -- 172.21.15.44:0/3991715561 <== mon.1 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2656026779 0 0) 0x7fceb0003520 con 0x7fcec81553f0 2024-04-22T06:31:36.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7fcec77fe640 1 -- 172.21.15.44:0/3991715561 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcec8155010 msgr2=0x7fcec815bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:36.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7fcec77fe640 1 --2- 172.21.15.44:0/3991715561 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcec8155010 0x7fcec815bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7fcec77fe640 1 -- 172.21.15.44:0/3991715561 >> v2:172.21.15.44:3300/0 conn(0x7fcec806d5a0 msgr2=0x7fcec80cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:36.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7fcec77fe640 1 --2- 172.21.15.44:0/3991715561 >> v2:172.21.15.44:3300/0 conn(0x7fcec806d5a0 0x7fcec80cbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7fcec77fe640 1 -- 172.21.15.44:0/3991715561 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec80c5d50 con 0x7fcec81553f0 2024-04-22T06:31:36.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7f8adcc0f640 1 --2- 172.21.15.44:0/1146833671 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ad8070300 0x7f8ad815f530 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0009e50 tx=0x7f8ad002be50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c36c95428dc785ab server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:36.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7fcec77fe640 1 -- 172.21.15.44:0/3991715561 <== mon.1 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1237339257 0 0) 0x7fcebc001650 con 0x7fcec81553f0 2024-04-22T06:31:36.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7fcec77fe640 1 -- 172.21.15.44:0/3991715561 >> v1:172.21.15.70:6789/0 conn(0x7fcec81553f0 legacy=0x7fcec80c6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7fa1b4dbf640 1 -- 172.21.15.44:0/2946745031 --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1b00c4740 con 0x7fa1b0068cc0 2024-04-22T06:31:36.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7fa1b4dbf640 1 -- 172.21.15.44:0/2946745031 --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fa1b0074580 con 0x7fa1b0151110 2024-04-22T06:31:36.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7fa1b4dbf640 1 -- 172.21.15.44:0/2946745031 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa1b00c4a20 con 0x7fa1b00c3d40 2024-04-22T06:31:36.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7fcec77fe640 1 --2- 172.21.15.44:0/3991715561 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fceb0003a10 0x7fceb0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7fcec77fe640 1 --2- 172.21.15.44:0/3991715561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fceb00043b0 0x7fceb0004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7fcec77fe640 1 --2- 172.21.15.44:0/3991715561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fceb0005030 0x7fceb000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7fcec77fe640 1 -- 172.21.15.44:0/3991715561 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcec80d1280 con 0x7fceb0005030 2024-04-22T06:31:36.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7fcec77fe640 1 -- 172.21.15.44:0/3991715561 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fcec80d1560 con 0x7fceb00043b0 2024-04-22T06:31:36.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7fcec77fe640 1 -- 172.21.15.44:0/3991715561 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fceb000bfb0 con 0x7fceb0003a10 2024-04-22T06:31:36.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7f09f37fe640 1 -- 172.21.15.44:0/3559714398 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e0034070 con 0x7f09ec04b500 2024-04-22T06:31:36.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7f09f37fe640 1 -- 172.21.15.44:0/3559714398 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09e0030860 con 0x7f09ec04b500 2024-04-22T06:31:36.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7f09f37fe640 1 -- 172.21.15.44:0/3559714398 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e0030c30 con 0x7f09ec04b500 2024-04-22T06:31:36.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.668+0000 7f09f9873640 1 -- 172.21.15.44:0/3559714398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09ec04b500 msgr2=0x7f09ec04b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.668+0000 7f09f9873640 1 --2- 172.21.15.44:0/3559714398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09ec04b500 0x7f09ec04b920 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f09ec0d5520 tx=0x7f09e002b9a0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.668+0000 7f09f9873640 1 -- 172.21.15.44:0/3559714398 shutdown_connections 2024-04-22T06:31:36.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.668+0000 7f09f9873640 1 --2- 172.21.15.44:0/3559714398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09ec04b500 0x7f09ec04b920 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.668+0000 7f09f9873640 1 --2- 172.21.15.44:0/3559714398 >> v2:172.21.15.44:3300/0 conn(0x7f09ec050ca0 0x7f09ec0dac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.668+0000 7f09f9873640 1 -- 172.21.15.44:0/3559714398 >> 172.21.15.44:0/3559714398 conn(0x7f09ec050180 msgr2=0x7f09ec050590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:36.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.668+0000 7f09f9873640 1 -- 172.21.15.44:0/3559714398 shutdown_connections 2024-04-22T06:31:36.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7f8acffff640 1 -- 172.21.15.44:0/1146833671 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad002c9f0 con 0x7f8ad8070300 2024-04-22T06:31:36.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.668+0000 7f8acffff640 1 -- 172.21.15.44:0/1146833671 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ad002cb90 con 0x7f8ad8070300 2024-04-22T06:31:36.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.668+0000 7fc924aa8640 1 -- 172.21.15.44:0/125025029 wait complete. 2024-04-22T06:31:36.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.668+0000 7fcec7fff640 1 --2- 172.21.15.44:0/3991715561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fceb00043b0 0x7fceb0004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.668+0000 7fceccbf8640 1 --2- 172.21.15.44:0/3991715561 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fceb0003a10 0x7fceb0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.668+0000 7fcecd3f9640 1 --2- 172.21.15.44:0/3991715561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fceb0005030 0x7fceb000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.668+0000 7fceccbf8640 1 -- 172.21.15.44:0/3991715561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fceb00043b0 msgr2=0x7fceb0004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:36.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.668+0000 7fceccbf8640 1 --2- 172.21.15.44:0/3991715561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fceb00043b0 0x7fceb0004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.668+0000 7fceccbf8640 1 -- 172.21.15.44:0/3991715561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fceb0005030 msgr2=0x7fceb000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:36.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.668+0000 7fceccbf8640 1 --2- 172.21.15.44:0/3991715561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fceb0005030 0x7fceb000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.668+0000 7fceccbf8640 1 -- 172.21.15.44:0/3991715561 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec80c5d50 con 0x7fceb0003a10 2024-04-22T06:31:36.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.668+0000 7fcecd3f9640 1 --2- 172.21.15.44:0/3991715561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fceb0005030 0x7fceb000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:31:36.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7f31467fc640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f314004ba90 0x7f31400e30c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:49458/0 (socket says 172.21.15.44:49458) 2024-04-22T06:31:36.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7f3146ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f31400dd3f0 0x7f3140046490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7f3146ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f31400dd3f0 0x7f3140046490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53516/0 (socket says 172.21.15.44:53516) 2024-04-22T06:31:36.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.668+0000 7f314d9ef640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31400e37d0 0x7f314004beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7f22ccd85640 1 -- 172.21.15.44:0/2612409532 --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22c815ccf0 con 0x7f22c81552e0 2024-04-22T06:31:36.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7f22ccd85640 1 -- 172.21.15.44:0/2612409532 --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f22c80c6740 con 0x7f22c8154f00 2024-04-22T06:31:36.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7f22ccd85640 1 -- 172.21.15.44:0/2612409532 --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f22c80c5b30 con 0x7f22c806d470 2024-04-22T06:31:36.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7f22c6d9d640 1 --2- 172.21.15.44:0/2612409532 >> v2:172.21.15.70:3300/0 conn(0x7f22c806d470 0x7f22c80d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7f22c659c640 1 -- 172.21.15.44:0/2612409532 >> v2:172.21.15.70:3300/0 conn(0x7f22c806d470 msgr2=0x7f22c80d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:36.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7f22c659c640 1 --2- 172.21.15.44:0/2612409532 >> v2:172.21.15.70:3300/0 conn(0x7f22c806d470 0x7f22c80d0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7f22c659c640 1 -- 172.21.15.44:0/2612409532 >> v1:172.21.15.44:6789/0 conn(0x7f22c81552e0 legacy=0x7f22c815c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7f22c659c640 1 -- 172.21.15.44:0/2612409532 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22c80c5e10 con 0x7f22c8154f00 2024-04-22T06:31:36.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.668+0000 7f8adf674640 1 -- 172.21.15.44:0/1146833671 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ad8070300 msgr2=0x7f8ad815f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.668+0000 7f8adf674640 1 --2- 172.21.15.44:0/1146833671 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ad8070300 0x7f8ad815f530 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0009e50 tx=0x7f8ad002be50 comp rx=0 tx=0).stop 2024-04-22T06:31:36.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.667+0000 7f3823e9a640 1 -- 172.21.15.44:0/1057060670 wait complete. 2024-04-22T06:31:36.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.669+0000 7f3823e9a640 1 Processor -- start 2024-04-22T06:31:36.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.669+0000 7f3823e9a640 1 -- start start 2024-04-22T06:31:36.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.668+0000 7f31467fc640 1 -- 172.21.15.44:0/3349989461 learned_addr learned my addr 172.21.15.44:0/3349989461 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:36.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.668+0000 7fa1affff640 1 -- 172.21.15.44:0/2946745031 >> v2:172.21.15.44:3300/0 conn(0x7fa1b0151110 msgr2=0x7fa1b0157990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:36.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.669+0000 7f314d9ef640 1 -- 172.21.15.44:0/3349989461 --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31400dd7d0 con 0x7f314004ba90 2024-04-22T06:31:36.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.669+0000 7f314d9ef640 1 -- 172.21.15.44:0/3349989461 --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f3140045480 con 0x7f31400dd3f0 2024-04-22T06:31:36.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.669+0000 7f314d9ef640 1 -- 172.21.15.44:0/3349989461 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3140046000 con 0x7f31400e37d0 2024-04-22T06:31:36.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.669+0000 7f31477fe640 1 --2- 172.21.15.44:0/3349989461 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31400e37d0 0x7f314004beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.668+0000 7fa1affff640 1 --2- 172.21.15.44:0/2946745031 >> v2:172.21.15.44:3300/0 conn(0x7fa1b0151110 0x7fa1b0157990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.668+0000 7fa1affff640 1 -- 172.21.15.44:0/2946745031 >> v1:172.21.15.70:6789/0 conn(0x7fa1b0068cc0 legacy=0x7fa1b00c4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:36.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.668+0000 7fa1affff640 1 -- 172.21.15.44:0/2946745031 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b006ec90 con 0x7fa1b00c3d40 2024-04-22T06:31:36.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.669+0000 7fa1af7fe640 1 --2- 172.21.15.44:0/2946745031 >> v2:172.21.15.44:3300/0 conn(0x7fa1b0151110 0x7fa1b0157990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:31:36.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.669+0000 7fa1affff640 1 --2- 172.21.15.44:0/2946745031 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa1b00c3d40 0x7fa1b00c8600 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8009dc0 tx=0x7fa1a802be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=664742da07d0922c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:36.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.669+0000 7f22c659c640 1 --2- 172.21.15.44:0/2612409532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22c8154f00 0x7f22c80cb430 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f22c806d190 tx=0x7f22b802b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfca827e915387a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:36.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.669+0000 7f8adf674640 1 -- 172.21.15.44:0/1146833671 shutdown_connections 2024-04-22T06:31:36.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.669+0000 7f8adf674640 1 --2- 172.21.15.44:0/1146833671 >> v2:172.21.15.44:3300/0 conn(0x7f8ad8074cb0 0x7f8ad8075140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.669+0000 7f8adf674640 1 --2- 172.21.15.44:0/1146833671 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ad8070300 0x7f8ad815f530 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.669+0000 7f8adf674640 1 -- 172.21.15.44:0/1146833671 >> 172.21.15.44:0/1146833671 conn(0x7f8ad80cd630 msgr2=0x7f8ad80cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:36.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.669+0000 7f22c5d9b640 1 -- 172.21.15.44:0/2612409532 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b8035020 con 0x7f22c8154f00 2024-04-22T06:31:36.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.669+0000 7f22c5d9b640 1 -- 172.21.15.44:0/2612409532 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22b8031880 con 0x7f22c8154f00 2024-04-22T06:31:36.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.669+0000 7f8adf674640 1 -- 172.21.15.44:0/1146833671 shutdown_connections 2024-04-22T06:31:36.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.669+0000 7f22c5d9b640 1 -- 172.21.15.44:0/2612409532 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b8031c50 con 0x7f22c8154f00 2024-04-22T06:31:36.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.670+0000 7fc924aa8640 1 Processor -- start 2024-04-22T06:31:36.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.670+0000 7f8adf674640 1 -- 172.21.15.44:0/1146833671 wait complete. 2024-04-22T06:31:36.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.670+0000 7fc924aa8640 1 -- start start 2024-04-22T06:31:36.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.670+0000 7fa1ae7fc640 1 -- 172.21.15.44:0/2946745031 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8039070 con 0x7fa1b00c3d40 2024-04-22T06:31:36.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.670+0000 7fa1ae7fc640 1 -- 172.21.15.44:0/2946745031 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1a802cbc0 con 0x7fa1b00c3d40 2024-04-22T06:31:36.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.670+0000 7fa1ae7fc640 1 -- 172.21.15.44:0/2946745031 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8034440 con 0x7fa1b00c3d40 2024-04-22T06:31:36.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.670+0000 7f3823e9a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3814052380 0x7f381404b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.668+0000 7f09f9873640 1 -- 172.21.15.44:0/3559714398 wait complete. 2024-04-22T06:31:36.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.670+0000 7f09f9873640 1 Processor -- start 2024-04-22T06:31:36.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.670+0000 7f3823e9a640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38140dd4f0 0x7f3814007c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.670+0000 7f3823e9a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38140e6f70 0x7f3814008770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.670+0000 7f3823e9a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f38140dd8d0 con 0x7f38140e6f70 2024-04-22T06:31:36.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.670+0000 7f3146ffd640 1 -- 172.21.15.44:0/3349989461 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31400e37d0 msgr2=0x7f314004beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:36.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.670+0000 7f3146ffd640 1 --2- 172.21.15.44:0/3349989461 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31400e37d0 0x7f314004beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.670+0000 7f3146ffd640 1 -- 172.21.15.44:0/3349989461 >> v1:172.21.15.44:6789/0 conn(0x7f314004ba90 legacy=0x7f31400e30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.670+0000 7f3146ffd640 1 -- 172.21.15.44:0/3349989461 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31400e3c40 con 0x7f31400dd3f0 2024-04-22T06:31:36.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.670+0000 7f3823e9a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f38140378b0 con 0x7f3814052380 2024-04-22T06:31:36.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.670+0000 7f3821c36640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3814052380 0x7f381404b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.670+0000 7f22ccd85640 1 -- 172.21.15.44:0/2612409532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22c8154f00 msgr2=0x7f22c80cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.670+0000 7f22ccd85640 1 --2- 172.21.15.44:0/2612409532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22c8154f00 0x7f22c80cb430 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f22c806d190 tx=0x7f22b802b920 comp rx=0 tx=0).stop 2024-04-22T06:31:36.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.671+0000 7f3822437640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38140e6f70 0x7f3814008770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.671+0000 7f3823e9a640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f38140dda50 con 0x7f38140dd4f0 2024-04-22T06:31:36.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.671+0000 7f3822437640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38140e6f70 0x7f3814008770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53560/0 (socket says 172.21.15.44:53560) 2024-04-22T06:31:36.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.671+0000 7f3822437640 1 -- 172.21.15.44:0/1195231814 learned_addr learned my addr 172.21.15.44:0/1195231814 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:36.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.671+0000 7fa1b4dbf640 1 -- 172.21.15.44:0/2946745031 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa1b00c3d40 msgr2=0x7fa1b00c8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.671+0000 7fa1b4dbf640 1 --2- 172.21.15.44:0/2946745031 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa1b00c3d40 0x7fa1b00c8600 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8009dc0 tx=0x7fa1a802be10 comp rx=0 tx=0).stop 2024-04-22T06:31:36.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.670+0000 7f09f9873640 1 -- start start 2024-04-22T06:31:36.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.671+0000 7f3822437640 1 -- 172.21.15.44:0/1195231814 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38140dd4f0 msgr2=0x7f3814007c40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-22T06:31:36.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.671+0000 7f3822437640 1 --2- 172.21.15.44:0/1195231814 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38140dd4f0 0x7f3814007c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.671+0000 7f3822437640 1 -- 172.21.15.44:0/1195231814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3814052380 msgr2=0x7f381404b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.671+0000 7f3822437640 1 --2- 172.21.15.44:0/1195231814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3814052380 0x7f381404b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.671+0000 7f3822437640 1 -- 172.21.15.44:0/1195231814 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38140495f0 con 0x7f38140e6f70 2024-04-22T06:31:36.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.671+0000 7fa1b4dbf640 1 -- 172.21.15.44:0/2946745031 shutdown_connections 2024-04-22T06:31:36.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.671+0000 7fa1b4dbf640 1 --2- 172.21.15.44:0/2946745031 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa1b00c3d40 0x7fa1b00c8600 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.671+0000 7fa1b4dbf640 1 --2- 172.21.15.44:0/2946745031 >> v2:172.21.15.44:3300/0 conn(0x7fa1b0151110 0x7fa1b0157990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.671+0000 7fa1b4dbf640 1 -- 172.21.15.44:0/2946745031 >> 172.21.15.44:0/2946745031 conn(0x7fa1b00cc840 msgr2=0x7fa1b00ccc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:36.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.671+0000 7fceccbf8640 1 --2- 172.21.15.44:0/3991715561 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fceb0003a10 0x7fceb0003e30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcec8073f30 tx=0x7fceb0005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23fbc7b53dfa0187 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:36.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.671+0000 7fa1b4dbf640 1 -- 172.21.15.44:0/2946745031 shutdown_connections 2024-04-22T06:31:36.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.671+0000 7fcec77fe640 1 -- 172.21.15.44:0/3991715561 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb8039070 con 0x7fceb0003a10 2024-04-22T06:31:36.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.671+0000 7fa1b4dbf640 1 -- 172.21.15.44:0/2946745031 wait complete. 2024-04-22T06:31:36.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.671+0000 7f8adf674640 1 Processor -- start 2024-04-22T06:31:36.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.671+0000 7fcec77fe640 1 -- 172.21.15.44:0/3991715561 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fceb802c580 con 0x7fceb0003a10 2024-04-22T06:31:36.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.671+0000 7fcec77fe640 1 -- 172.21.15.44:0/3991715561 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb8009650 con 0x7fceb0003a10 2024-04-22T06:31:36.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.671+0000 7f3822437640 1 --2- 172.21.15.44:0/1195231814 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38140e6f70 0x7f3814008770 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f381c000b60 tx=0x7f381c0aa070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:36.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.671+0000 7f8adf674640 1 -- start start 2024-04-22T06:31:36.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.671+0000 7fc924aa8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc920070300 0x7fc920151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.672+0000 7f22ccd85640 1 -- 172.21.15.44:0/2612409532 shutdown_connections 2024-04-22T06:31:36.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.672+0000 7f22ccd85640 1 --2- 172.21.15.44:0/2612409532 >> v2:172.21.15.70:3300/0 conn(0x7f22c806d470 0x7f22c80d0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.672+0000 7f22ccd85640 1 --2- 172.21.15.44:0/2612409532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22c8154f00 0x7f22c80cb430 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.672+0000 7f22ccd85640 1 -- 172.21.15.44:0/2612409532 >> 172.21.15.44:0/2612409532 conn(0x7f22c80733c0 msgr2=0x7f22c80737d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:36.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.672+0000 7fc924aa8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9201595b0 0x7fc9201577d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.672+0000 7f38127fc640 1 -- 172.21.15.44:0/1195231814 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381c0c0020 con 0x7f38140e6f70 2024-04-22T06:31:36.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.672+0000 7f38127fc640 1 -- 172.21.15.44:0/1195231814 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f381c0ba880 con 0x7f38140e6f70 2024-04-22T06:31:36.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.672+0000 7f09f9873640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09ec005ee0 0x7f09ec0d2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.672+0000 7f09f9873640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f09ec04b500 0x7f09ec0d7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.672+0000 7f09f9873640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09ec050ca0 0x7f09ec0ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.672+0000 7f09f9873640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f09ec0d5000 con 0x7f09ec005ee0 2024-04-22T06:31:36.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.672+0000 7f09f9873640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f09ec007480 con 0x7f09ec050ca0 2024-04-22T06:31:36.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.672+0000 7f09f9873640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f09ec0d5180 con 0x7f09ec04b500 2024-04-22T06:31:36.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.672+0000 7f38127fc640 1 -- 172.21.15.44:0/1195231814 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381c0b5040 con 0x7f38140e6f70 2024-04-22T06:31:36.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.672+0000 7f3823e9a640 1 -- 172.21.15.44:0/1195231814 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3814049870 con 0x7f38140e6f70 2024-04-22T06:31:36.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.672+0000 7fc924aa8640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc92015f530 0x7fc920157d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.672+0000 7fc91e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc920070300 0x7fc920151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.672+0000 7fc91e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc920070300 0x7fc920151f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53562/0 (socket says 172.21.15.44:53562) 2024-04-22T06:31:36.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.672+0000 7fc91dd9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9201595b0 0x7fc9201577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.672+0000 7fc91dd9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9201595b0 0x7fc9201577d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:34062/0 (socket says 172.21.15.44:34062) 2024-04-22T06:31:36.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.672+0000 7f3823e9a640 1 -- 172.21.15.44:0/1195231814 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f381400a2c0 con 0x7f38140e6f70 2024-04-22T06:31:36.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.672+0000 7fcecee5c640 1 -- 172.21.15.44:0/3991715561 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fceb0003a10 msgr2=0x7fceb0003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.672+0000 7fcecee5c640 1 --2- 172.21.15.44:0/3991715561 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fceb0003a10 0x7fceb0003e30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcec8073f30 tx=0x7fceb0005830 comp rx=0 tx=0).stop 2024-04-22T06:31:36.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.672+0000 7f09f8871640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09ec005ee0 0x7f09ec0d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.672+0000 7f09f8871640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09ec005ee0 0x7f09ec0d2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53564/0 (socket says 172.21.15.44:53564) 2024-04-22T06:31:36.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.672+0000 7f09f8871640 1 -- 172.21.15.44:0/1241774191 learned_addr learned my addr 172.21.15.44:0/1241774191 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:36.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.672+0000 7f09f3fff640 1 --2- 172.21.15.44:0/1241774191 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f09ec04b500 0x7f09ec0d7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.673+0000 7f8adf674640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ad8070300 0x7f8ad81458a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.672+0000 7f09f8871640 1 -- 172.21.15.44:0/1241774191 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f09ec04b500 msgr2=0x7f09ec0d7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.672+0000 7f09f8871640 1 --2- 172.21.15.44:0/1241774191 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f09ec04b500 0x7f09ec0d7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.672+0000 7f09f8871640 1 -- 172.21.15.44:0/1241774191 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09ec050ca0 msgr2=0x7f09ec0ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.672+0000 7f09f8871640 1 --2- 172.21.15.44:0/1241774191 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09ec050ca0 0x7f09ec0ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.673+0000 7f09f8871640 1 -- 172.21.15.44:0/1241774191 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09ec181780 con 0x7f09ec005ee0 2024-04-22T06:31:36.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.673+0000 7f8adf674640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ad8074cb0 0x7f8ad80ca650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.673+0000 7f09f8871640 1 --2- 172.21.15.44:0/1241774191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09ec005ee0 0x7f09ec0d2680 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f09e800d800 tx=0x7f09e800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:36.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.673+0000 7fa1b4dbf640 1 Processor -- start 2024-04-22T06:31:36.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.673+0000 7f22ccd85640 1 -- 172.21.15.44:0/2612409532 shutdown_connections 2024-04-22T06:31:36.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.673+0000 7f22ccd85640 1 -- 172.21.15.44:0/2612409532 wait complete. 2024-04-22T06:31:36.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.672+0000 7fc924aa8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc920074290 con 0x7fc920070300 2024-04-22T06:31:36.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.672+0000 7fc924aa8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc920159990 con 0x7fc9201595b0 2024-04-22T06:31:36.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.672+0000 7fc924aa8640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc9200ce470 con 0x7fc92015f530 2024-04-22T06:31:36.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.673+0000 7fc91e59c640 1 -- 172.21.15.44:0/3788021894 learned_addr learned my addr 172.21.15.44:0/3788021894 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:36.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.673+0000 7fc91ed9d640 1 --2- 172.21.15.44:0/3788021894 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc92015f530 0x7fc920157d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.673+0000 7f3823e9a640 1 -- 172.21.15.44:0/1195231814 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3804001030 con 0x7f38140e6f70 2024-04-22T06:31:36.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.673+0000 7f8adf674640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ad81595b0 0x7f8ad80cab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.673+0000 7fcecd3f9640 1 -- 172.21.15.44:0/3991715561 reap_dead start 2024-04-22T06:31:36.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.673+0000 7f8adf674640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad8074290 con 0x7f8ad8074cb0 2024-04-22T06:31:36.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.673+0000 7f8adf674640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad8159990 con 0x7f8ad81595b0 2024-04-22T06:31:36.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.673+0000 7f8adf674640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8ad80ce470 con 0x7f8ad8070300 2024-04-22T06:31:36.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.673+0000 7fa1b4dbf640 1 -- start start 2024-04-22T06:31:36.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.673+0000 7f8add410640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ad8070300 0x7f8ad81458a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.673+0000 7f8add410640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ad8070300 0x7f8ad81458a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.44:45498/0 (socket says 172.21.15.44:45498) 2024-04-22T06:31:36.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.673+0000 7f8add410640 1 -- 172.21.15.44:0/4179939059 learned_addr learned my addr 172.21.15.44:0/4179939059 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:36.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.673+0000 7f8addc11640 1 --2- 172.21.15.44:0/4179939059 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ad81595b0 0x7f8ad80cab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.673+0000 7fcecee5c640 1 -- 172.21.15.44:0/3991715561 shutdown_connections 2024-04-22T06:31:36.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.673+0000 7fcecee5c640 1 -- 172.21.15.44:0/3991715561 >> 172.21.15.44:0/3991715561 conn(0x7fcec80750d0 msgr2=0x7fcec80754e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:36.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.673+0000 7fcecee5c640 1 -- 172.21.15.44:0/3991715561 shutdown_connections 2024-04-22T06:31:36.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.674+0000 7f09f17fa640 1 -- 172.21.15.44:0/1241774191 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e8004030 con 0x7f09ec005ee0 2024-04-22T06:31:36.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.674+0000 7f09f9873640 1 -- 172.21.15.44:0/1241774191 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09ec0d5b80 con 0x7f09ec005ee0 2024-04-22T06:31:36.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.674+0000 7f09f17fa640 1 -- 172.21.15.44:0/1241774191 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09e80041d0 con 0x7f09ec005ee0 2024-04-22T06:31:36.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.674+0000 7f8add410640 1 -- 172.21.15.44:0/4179939059 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ad81595b0 msgr2=0x7f8ad80cab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.674+0000 7f8add410640 1 --2- 172.21.15.44:0/4179939059 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ad81595b0 0x7f8ad80cab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.674+0000 7f8add410640 1 -- 172.21.15.44:0/4179939059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ad8074cb0 msgr2=0x7f8ad80ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.674+0000 7f8add410640 1 --2- 172.21.15.44:0/4179939059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ad8074cb0 0x7f8ad80ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.674+0000 7f8add410640 1 -- 172.21.15.44:0/4179939059 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad8143d10 con 0x7f8ad8070300 2024-04-22T06:31:36.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.674+0000 7fc91dd9b640 1 -- 172.21.15.44:0/3788021894 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc92015f530 msgr2=0x7fc920157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.674+0000 7fc91dd9b640 1 --2- 172.21.15.44:0/3788021894 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc92015f530 0x7fc920157d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.674+0000 7fc91dd9b640 1 -- 172.21.15.44:0/3788021894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc920070300 msgr2=0x7fc920151f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.674+0000 7fc91dd9b640 1 --2- 172.21.15.44:0/3788021894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc920070300 0x7fc920151f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.674+0000 7fc91dd9b640 1 -- 172.21.15.44:0/3788021894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc920155320 con 0x7fc9201595b0 2024-04-22T06:31:36.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.674+0000 7f09f17fa640 1 -- 172.21.15.44:0/1241774191 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e8004030 con 0x7f09ec005ee0 2024-04-22T06:31:36.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.674+0000 7fc91e59c640 1 --2- 172.21.15.44:0/3788021894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc920070300 0x7fc920151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:31:36.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.674+0000 7f8add410640 1 --2- 172.21.15.44:0/4179939059 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ad8070300 0x7f8ad81458a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400c410 tx=0x7f8ad400c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:36.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.674+0000 7fc91dd9b640 1 --2- 172.21.15.44:0/3788021894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9201595b0 0x7fc9201577d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc91000c780 tx=0x7fc91000cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:36.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.674+0000 7f09f9873640 1 -- 172.21.15.44:0/1241774191 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09ec0d58a0 con 0x7f09ec005ee0 2024-04-22T06:31:36.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.674+0000 7fa1b4dbf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1b0068cc0 0x7fa1b0154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.674+0000 7fa1b4dbf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1b00c3d40 0x7fa1b0155440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.674+0000 7fa1b4dbf640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa1b0151110 0x7fa1b0141f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.674+0000 7fa1b4dbf640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b0157ed0 con 0x7fa1b00c3d40 2024-04-22T06:31:36.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.674+0000 7fa1b4dbf640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b006f5c0 con 0x7fa1b0068cc0 2024-04-22T06:31:36.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.674+0000 7fa1b4dbf640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa1b006f250 con 0x7fa1b0151110 2024-04-22T06:31:36.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.675+0000 7fa1affff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa1b0151110 0x7fa1b0141f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.675+0000 7fc90effd640 1 -- 172.21.15.44:0/3788021894 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc910005270 con 0x7fc9201595b0 2024-04-22T06:31:36.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.675+0000 7fc90effd640 1 -- 172.21.15.44:0/3788021894 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc910005410 con 0x7fc9201595b0 2024-04-22T06:31:36.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.675+0000 7fc90effd640 1 -- 172.21.15.44:0/3788021894 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91000a860 con 0x7fc9201595b0 2024-04-22T06:31:36.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.675+0000 7fa1affff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa1b0151110 0x7fa1b0141f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.44:45510/0 (socket says 172.21.15.44:45510) 2024-04-22T06:31:36.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.675+0000 7fa1affff640 1 -- 172.21.15.44:0/2287425767 learned_addr learned my addr 172.21.15.44:0/2287425767 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:36.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.675+0000 7fa1af7fe640 1 --2- 172.21.15.44:0/2287425767 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1b0068cc0 0x7fa1b0154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.675+0000 7f8acdffb640 1 -- 172.21.15.44:0/4179939059 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4017070 con 0x7f8ad8070300 2024-04-22T06:31:36.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.675+0000 7f8acdffb640 1 -- 172.21.15.44:0/4179939059 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ad40035f0 con 0x7f8ad8070300 2024-04-22T06:31:36.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.675+0000 7f8acdffb640 1 -- 172.21.15.44:0/4179939059 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4003a20 con 0x7f8ad8070300 2024-04-22T06:31:36.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.675+0000 7f8adf674640 1 -- 172.21.15.44:0/4179939059 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ad8143ff0 con 0x7f8ad8070300 2024-04-22T06:31:36.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.675+0000 7f22ccd85640 1 Processor -- start 2024-04-22T06:31:36.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.676+0000 7f22ccd85640 1 -- start start 2024-04-22T06:31:36.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.676+0000 7f3146ffd640 1 --2- 172.21.15.44:0/3349989461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31400dd3f0 0x7f3140046490 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3138009a00 tx=0x7f313802bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bebcc03638ee4418 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:36.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.676+0000 7f09f9873640 1 -- 172.21.15.44:0/1241774191 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f09ec0d2bc0 con 0x7f09ec005ee0 2024-04-22T06:31:36.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.676+0000 7f8adf674640 1 -- 172.21.15.44:0/4179939059 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ad8144890 con 0x7f8ad8070300 2024-04-22T06:31:36.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.676+0000 7fa1affff640 1 -- 172.21.15.44:0/2287425767 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1b0068cc0 msgr2=0x7fa1b0154f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.676+0000 7fa1affff640 1 --2- 172.21.15.44:0/2287425767 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1b0068cc0 0x7fa1b0154f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.676+0000 7fa1affff640 1 -- 172.21.15.44:0/2287425767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1b00c3d40 msgr2=0x7fa1b0155440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.676+0000 7fa1affff640 1 --2- 172.21.15.44:0/2287425767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1b00c3d40 0x7fa1b0155440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.676+0000 7fa1affff640 1 -- 172.21.15.44:0/2287425767 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b0145cf0 con 0x7fa1b0151110 2024-04-22T06:31:36.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.676+0000 7fa1affff640 1 --2- 172.21.15.44:0/2287425767 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa1b0151110 0x7fa1b0141f40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8009ee0 tx=0x7fa1a8004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:36.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.676+0000 7fa197fff640 1 -- 172.21.15.44:0/2287425767 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8042020 con 0x7fa1b0151110 2024-04-22T06:31:36.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.676+0000 7fa197fff640 1 -- 172.21.15.44:0/2287425767 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1a80042f0 con 0x7fa1b0151110 2024-04-22T06:31:36.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.676+0000 7fa1b4dbf640 1 -- 172.21.15.44:0/2287425767 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1b0145fd0 con 0x7fa1b0151110 2024-04-22T06:31:36.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.677+0000 7fa197fff640 1 -- 172.21.15.44:0/2287425767 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8039040 con 0x7fa1b0151110 2024-04-22T06:31:36.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.677+0000 7fa1b4dbf640 1 -- 172.21.15.44:0/2287425767 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1b01572f0 con 0x7fa1b0151110 2024-04-22T06:31:36.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.676+0000 7f38127fc640 1 -- 172.21.15.44:0/1195231814 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f381c0c3020 con 0x7f38140e6f70 2024-04-22T06:31:36.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.677+0000 7f22ccd85640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f22c806d470 0x7f22c80cd580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.677+0000 7f22ccd85640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22c8154f00 0x7f22c815d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.677+0000 7f22ccd85640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22c81552e0 0x7f22c815d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.677+0000 7f22ccd85640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f22c80c6bd0 con 0x7f22c8154f00 2024-04-22T06:31:36.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.677+0000 7f22ccd85640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f22c80c63d0 con 0x7f22c81552e0 2024-04-22T06:31:36.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.677+0000 7f22ccd85640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f22c80c6d50 con 0x7f22c806d470 2024-04-22T06:31:36.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.674+0000 7fcecee5c640 1 -- 172.21.15.44:0/3991715561 wait complete. 2024-04-22T06:31:36.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.675+0000 7fcecee5c640 1 Processor -- start 2024-04-22T06:31:36.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.675+0000 7fcecee5c640 1 -- start start 2024-04-22T06:31:36.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.677+0000 7fcecee5c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcec81500b0 0x7fcec8152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.677+0000 7fcecee5c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcec8155960 0x7fcec8149b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.677+0000 7f38127fc640 1 --2- 172.21.15.44:0/1195231814 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f380006c8f0 0x7f380006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.677+0000 7f22c659c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f22c806d470 0x7f22c80cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.677+0000 7f8abb7fe640 1 -- 172.21.15.44:0/4179939059 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8ad81443f0 con 0x7f8ad8070300 2024-04-22T06:31:36.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.677+0000 7f38127fc640 1 -- 172.21.15.44:0/1195231814 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f381c12ad80 con 0x7f38140e6f70 2024-04-22T06:31:36.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.677+0000 7f22c659c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f22c806d470 0x7f22c80cd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.44:45522/0 (socket says 172.21.15.44:45522) 2024-04-22T06:31:36.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.677+0000 7fcecee5c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcec813fc70 0x7fcec8150490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.677+0000 7fcecee5c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcec80cb450 con 0x7fcec8155960 2024-04-22T06:31:36.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.677+0000 7fcecee5c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fcec80cb2d0 con 0x7fcec813fc70 2024-04-22T06:31:36.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.677+0000 7fcecee5c640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fcec80cc6f0 con 0x7fcec81500b0 2024-04-22T06:31:36.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.677+0000 7fceccbf8640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcec81500b0 0x7fcec8152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.677+0000 7fceccbf8640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcec81500b0 0x7fcec8152b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.44:45526/0 (socket says 172.21.15.44:45526) 2024-04-22T06:31:36.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.677+0000 7fceccbf8640 1 -- 172.21.15.44:0/1198105874 learned_addr learned my addr 172.21.15.44:0/1198105874 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:36.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.677+0000 7f22c659c640 1 -- 172.21.15.44:0/3432976022 learned_addr learned my addr 172.21.15.44:0/3432976022 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:36.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.677+0000 7f3821c36640 1 --2- 172.21.15.44:0/1195231814 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f380006c8f0 0x7f380006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.678+0000 7f38127fc640 1 --2- 172.21.15.44:0/1195231814 >> [v2:172.21.15.70:6818/3802577427,v1:172.21.15.70:6820/3802577427] conn(0x7f3800075f60 0x7f38000783a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.677+0000 7f09f17fa640 1 -- 172.21.15.44:0/1241774191 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f09e8004380 con 0x7f09ec005ee0 2024-04-22T06:31:36.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.678+0000 7f38127fc640 1 -- 172.21.15.44:0/1195231814 --> [v2:172.21.15.70:6818/3802577427,v1:172.21.15.70:6820/3802577427] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f381c06c790 con 0x7f3800075f60 2024-04-22T06:31:36.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.678+0000 7f38127fc640 1 -- 172.21.15.44:0/1195231814 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f381c0f4aa0 con 0x7f38140e6f70 2024-04-22T06:31:36.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.678+0000 7fa1b4dbf640 1 -- 172.21.15.44:0/2287425767 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa1b00761b0 con 0x7fa1b0151110 2024-04-22T06:31:36.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.678+0000 7fc924aa8640 1 -- 172.21.15.44:0/3788021894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc920155600 con 0x7fc9201595b0 2024-04-22T06:31:36.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.677+0000 7f22c6d9d640 1 --2- 172.21.15.44:0/3432976022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22c81552e0 0x7f22c815d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.677+0000 7f22bdd9b640 1 --2- 172.21.15.44:0/3432976022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22c8154f00 0x7f22c815d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.678+0000 7fc924aa8640 1 -- 172.21.15.44:0/3788021894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc920156fd0 con 0x7fc9201595b0 2024-04-22T06:31:36.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.678+0000 7f3145ffb640 1 -- 172.21.15.44:0/3349989461 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313802d6c0 con 0x7f31400dd3f0 2024-04-22T06:31:36.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.678+0000 7f3145ffb640 1 -- 172.21.15.44:0/3349989461 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f313802d860 con 0x7f31400dd3f0 2024-04-22T06:31:36.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.678+0000 7f3145ffb640 1 -- 172.21.15.44:0/3349989461 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313802dc30 con 0x7f31400dd3f0 2024-04-22T06:31:36.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.678+0000 7f22c659c640 1 -- 172.21.15.44:0/3432976022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22c81552e0 msgr2=0x7f22c815d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.678+0000 7f22c659c640 1 --2- 172.21.15.44:0/3432976022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22c81552e0 0x7f22c815d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.678+0000 7f22c659c640 1 -- 172.21.15.44:0/3432976022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22c8154f00 msgr2=0x7f22c815d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.678+0000 7f22c659c640 1 --2- 172.21.15.44:0/3432976022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22c8154f00 0x7f22c815d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.678+0000 7f22c659c640 1 -- 172.21.15.44:0/3432976022 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22c815dbd0 con 0x7f22c806d470 2024-04-22T06:31:36.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.677+0000 7fcec7fff640 1 --2- 172.21.15.44:0/1198105874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcec8155960 0x7fcec8149b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.677+0000 7fceccbf8640 1 -- 172.21.15.44:0/1198105874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcec813fc70 msgr2=0x7fcec8150490 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-22T06:31:36.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.677+0000 7fceccbf8640 1 --2- 172.21.15.44:0/1198105874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcec813fc70 0x7fcec8150490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.677+0000 7fceccbf8640 1 -- 172.21.15.44:0/1198105874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcec8155960 msgr2=0x7fcec8149b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.678+0000 7fceccbf8640 1 --2- 172.21.15.44:0/1198105874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcec8155960 0x7fcec8149b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.678+0000 7fceccbf8640 1 -- 172.21.15.44:0/1198105874 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec8146920 con 0x7fcec81500b0 2024-04-22T06:31:36.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.678+0000 7f3821435640 1 --2- 172.21.15.44:0/1195231814 >> [v2:172.21.15.70:6818/3802577427,v1:172.21.15.70:6820/3802577427] conn(0x7f3800075f60 0x7f38000783a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.678+0000 7fcec7fff640 1 --2- 172.21.15.44:0/1198105874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcec8155960 0x7fcec8149b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:31:36.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.678+0000 7fceccbf8640 1 --2- 172.21.15.44:0/1198105874 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcec81500b0 0x7fcec8152b20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fceb8009a40 tx=0x7fceb8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:36.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.678+0000 7f22c659c640 1 --2- 172.21.15.44:0/3432976022 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f22c806d470 0x7f22c80cd580 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f22b8000c00 tx=0x7f22b80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:36.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.678+0000 7f3821c36640 1 --2- 172.21.15.44:0/1195231814 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f380006c8f0 0x7f380006edb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f380c00a810 tx=0x7f380c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:36.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.678+0000 7f09f17fa640 1 --2- 172.21.15.44:0/1241774191 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f09d806ccf0 0x7f09d806f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.678+0000 7f22bf7fe640 1 -- 172.21.15.44:0/3432976022 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b8040070 con 0x7f22c806d470 2024-04-22T06:31:36.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.679+0000 7f09f17fa640 1 -- 172.21.15.44:0/1241774191 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f09e800f040 con 0x7f09ec005ee0 2024-04-22T06:31:36.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.679+0000 7f09f3fff640 1 --2- 172.21.15.44:0/1241774191 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f09d806ccf0 0x7f09d806f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.679+0000 7f314d9ef640 1 -- 172.21.15.44:0/3349989461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31400dd3f0 msgr2=0x7f3140046490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.679+0000 7f314d9ef640 1 --2- 172.21.15.44:0/3349989461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31400dd3f0 0x7f3140046490 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3138009a00 tx=0x7f313802bb10 comp rx=0 tx=0).stop 2024-04-22T06:31:36.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.679+0000 7f09f17fa640 1 --2- 172.21.15.44:0/1241774191 >> [v2:172.21.15.44:6808/1278678977,v1:172.21.15.44:6809/1278678977] conn(0x7f09d8076360 0x7f09d80787a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.679+0000 7f314d9ef640 1 -- 172.21.15.44:0/3349989461 shutdown_connections 2024-04-22T06:31:36.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.679+0000 7f314d9ef640 1 --2- 172.21.15.44:0/3349989461 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31400e37d0 0x7f314004beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.679+0000 7f314d9ef640 1 --2- 172.21.15.44:0/3349989461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31400dd3f0 0x7f3140046490 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.679+0000 7f314d9ef640 1 -- 172.21.15.44:0/3349989461 >> 172.21.15.44:0/3349989461 conn(0x7f31400501b0 msgr2=0x7f31400505c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:36.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.679+0000 7f09f17fa640 1 -- 172.21.15.44:0/1241774191 --> [v2:172.21.15.44:6808/1278678977,v1:172.21.15.44:6809/1278678977] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f09e801ae70 con 0x7f09d8076360 2024-04-22T06:31:36.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.679+0000 7f09f17fa640 1 -- 172.21.15.44:0/1241774191 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f09e80902c0 con 0x7f09ec005ee0 2024-04-22T06:31:36.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.679+0000 7fcec57fa640 1 -- 172.21.15.44:0/1198105874 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb8043020 con 0x7fcec81500b0 2024-04-22T06:31:36.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.679+0000 7f09f9072640 1 --2- 172.21.15.44:0/1241774191 >> [v2:172.21.15.44:6808/1278678977,v1:172.21.15.44:6809/1278678977] conn(0x7f09d8076360 0x7f09d80787a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.679+0000 7fcecee5c640 1 -- 172.21.15.44:0/1198105874 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcec815f000 con 0x7fcec81500b0 2024-04-22T06:31:36.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.679+0000 7fcecee5c640 1 -- 172.21.15.44:0/1198105874 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcec81490a0 con 0x7fcec81500b0 2024-04-22T06:31:36.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.679+0000 7fcec57fa640 1 -- 172.21.15.44:0/1198105874 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fceb802dcc0 con 0x7fcec81500b0 2024-04-22T06:31:36.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.679+0000 7f314d9ef640 1 -- 172.21.15.44:0/3349989461 shutdown_connections 2024-04-22T06:31:36.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.679+0000 7fcec57fa640 1 -- 172.21.15.44:0/1198105874 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb8039040 con 0x7fcec81500b0 2024-04-22T06:31:36.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.679+0000 7f314d9ef640 1 -- 172.21.15.44:0/3349989461 wait complete. 2024-04-22T06:31:36.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.679+0000 7f22ccd85640 1 -- 172.21.15.44:0/3432976022 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22c81f0f40 con 0x7f22c806d470 2024-04-22T06:31:36.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.679+0000 7fc924aa8640 1 -- 172.21.15.44:0/3788021894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc920069310 con 0x7fc9201595b0 2024-04-22T06:31:36.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.680+0000 7f09f9072640 1 --2- 172.21.15.44:0/1241774191 >> [v2:172.21.15.44:6808/1278678977,v1:172.21.15.44:6809/1278678977] conn(0x7f09d8076360 0x7f09d80787a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:36.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.679+0000 7f3821435640 1 --2- 172.21.15.44:0/1195231814 >> [v2:172.21.15.70:6818/3802577427,v1:172.21.15.70:6820/3802577427] conn(0x7f3800075f60 0x7f38000783a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:36.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.679+0000 7f22ccd85640 1 -- 172.21.15.44:0/3432976022 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f22c814fd90 con 0x7f22c806d470 2024-04-22T06:31:36.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.680+0000 7f22ccd85640 1 -- 172.21.15.44:0/3432976022 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f22c80cbcd0 con 0x7f22c806d470 2024-04-22T06:31:36.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.680+0000 7f22bf7fe640 1 -- 172.21.15.44:0/3432976022 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22b803a630 con 0x7f22c806d470 2024-04-22T06:31:36.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.680+0000 7f22bf7fe640 1 -- 172.21.15.44:0/3432976022 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b8035040 con 0x7f22c806d470 2024-04-22T06:31:36.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.680+0000 7fc90effd640 1 -- 172.21.15.44:0/3788021894 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc91000aa00 con 0x7fc9201595b0 2024-04-22T06:31:36.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.680+0000 7f38127fc640 1 -- 172.21.15.44:0/1195231814 <== osd.4 v2:172.21.15.70:6818/3802577427 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f381c06c790 con 0x7f3800075f60 2024-04-22T06:31:36.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.681+0000 7fcecee5c640 1 -- 172.21.15.44:0/1198105874 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcea8001030 con 0x7fcec81500b0 2024-04-22T06:31:36.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.681+0000 7fa197fff640 1 -- 172.21.15.44:0/2287425767 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa1a8045020 con 0x7fa1b0151110 2024-04-22T06:31:36.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.680+0000 7f8acdffb640 1 -- 172.21.15.44:0/4179939059 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8ad401a070 con 0x7f8ad8070300 2024-04-22T06:31:36.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.680+0000 7f8acdffb640 1 --2- 172.21.15.44:0/4179939059 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f8abc06c970 0x7f8abc06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.680+0000 7f8adcc0f640 1 --2- 172.21.15.44:0/4179939059 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f8abc06c970 0x7f8abc06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.681+0000 7f8acdffb640 1 -- 172.21.15.44:0/4179939059 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8ad4056f20 con 0x7f8ad8070300 2024-04-22T06:31:36.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.681+0000 7fc90effd640 1 --2- 172.21.15.44:0/3788021894 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fc90006c9f0 0x7fc90006eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.681+0000 7f09f3fff640 1 --2- 172.21.15.44:0/1241774191 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f09d806ccf0 0x7f09d806f1b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f09e0036000 tx=0x7f09e0008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:36.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.681+0000 7f8acdffb640 1 --2- 172.21.15.44:0/4179939059 >> [v2:172.21.15.44:6800/2544087903,v1:172.21.15.44:6801/2544087903] conn(0x7f8abc076d40 0x7f8abc0791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.681+0000 7f8acdffb640 1 -- 172.21.15.44:0/4179939059 --> [v2:172.21.15.44:6800/2544087903,v1:172.21.15.44:6801/2544087903] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8abc0798b0 con 0x7f8abc076d40 2024-04-22T06:31:36.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.681+0000 7fc91e59c640 1 --2- 172.21.15.44:0/3788021894 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fc90006c9f0 0x7fc90006eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.681+0000 7fc90effd640 1 -- 172.21.15.44:0/3788021894 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc91008bc00 con 0x7fc9201595b0 2024-04-22T06:31:36.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.681+0000 7f8acdffb640 1 -- 172.21.15.44:0/4179939059 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8ad40558b0 con 0x7f8ad8070300 2024-04-22T06:31:36.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.681+0000 7fa197fff640 1 --2- 172.21.15.44:0/2287425767 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fa19c06c8f0 0x7fa19c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.681+0000 7f8adcc0f640 1 --2- 172.21.15.44:0/4179939059 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f8abc06c970 0x7f8abc06ee30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0035000 tx=0x7f8ad002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:36.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.681+0000 7fc90effd640 1 --2- 172.21.15.44:0/3788021894 >> [v2:172.21.15.44:6824/1692928109,v1:172.21.15.44:6825/1692928109] conn(0x7fc9000760e0 0x7fc900078520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.681+0000 7f314d9ef640 1 Processor -- start 2024-04-22T06:31:36.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.682+0000 7f314d9ef640 1 -- start start 2024-04-22T06:31:36.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.681+0000 7fa197fff640 1 -- 172.21.15.44:0/2287425767 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa1a80ae410 con 0x7fa1b0151110 2024-04-22T06:31:36.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.682+0000 7fa197fff640 1 --2- 172.21.15.44:0/2287425767 >> [v2:172.21.15.44:6816/2220268493,v1:172.21.15.44:6817/2220268493] conn(0x7fa19c076060 0x7fa19c0784a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.682+0000 7fa197fff640 1 -- 172.21.15.44:0/2287425767 --> [v2:172.21.15.44:6816/2220268493,v1:172.21.15.44:6817/2220268493] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa19c078b70 con 0x7fa19c076060 2024-04-22T06:31:36.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.682+0000 7fa197fff640 1 -- 172.21.15.44:0/2287425767 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa1a80ae800 con 0x7fa1b0151110 2024-04-22T06:31:36.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.682+0000 7fa1aeffd640 1 --2- 172.21.15.44:0/2287425767 >> [v2:172.21.15.44:6816/2220268493,v1:172.21.15.44:6817/2220268493] conn(0x7fa19c076060 0x7fa19c0784a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.682+0000 7f09f17fa640 1 -- 172.21.15.44:0/1241774191 <== osd.1 v2:172.21.15.44:6808/1278678977 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f09e801ae70 con 0x7f09d8076360 2024-04-22T06:31:36.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.682+0000 7fc90effd640 1 -- 172.21.15.44:0/3788021894 --> [v2:172.21.15.44:6824/1692928109,v1:172.21.15.44:6825/1692928109] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc900078bf0 con 0x7fc9000760e0 2024-04-22T06:31:36.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.682+0000 7fc91ed9d640 1 --2- 172.21.15.44:0/3788021894 >> [v2:172.21.15.44:6824/1692928109,v1:172.21.15.44:6825/1692928109] conn(0x7fc9000760e0 0x7fc900078520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.682+0000 7fc90effd640 1 -- 172.21.15.44:0/3788021894 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc910055890 con 0x7fc9201595b0 2024-04-22T06:31:36.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.682+0000 7fc91ed9d640 1 --2- 172.21.15.44:0/3788021894 >> [v2:172.21.15.44:6824/1692928109,v1:172.21.15.44:6825/1692928109] conn(0x7fc9000760e0 0x7fc900078520 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:36.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.682+0000 7f22bf7fe640 1 -- 172.21.15.44:0/3432976022 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f22b8038020 con 0x7f22c806d470 2024-04-22T06:31:36.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.682+0000 7fa1aeffd640 1 --2- 172.21.15.44:0/2287425767 >> [v2:172.21.15.44:6816/2220268493,v1:172.21.15.44:6817/2220268493] conn(0x7fa19c076060 0x7fa19c0784a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:36.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.682+0000 7fcec57fa640 1 -- 172.21.15.44:0/1198105874 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fceb8046020 con 0x7fcec81500b0 2024-04-22T06:31:36.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.682+0000 7fcec57fa640 1 --2- 172.21.15.44:0/1198105874 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fceb406c9c0 0x7fceb406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.682+0000 7fcec57fa640 1 -- 172.21.15.44:0/1198105874 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fceb80acff0 con 0x7fcec81500b0 2024-04-22T06:31:36.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.683+0000 7f8addc11640 1 --2- 172.21.15.44:0/4179939059 >> [v2:172.21.15.44:6800/2544087903,v1:172.21.15.44:6801/2544087903] conn(0x7f8abc076d40 0x7f8abc0791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.683+0000 7fcec57fa640 1 --2- 172.21.15.44:0/1198105874 >> [v2:172.21.15.70:6800/664757345,v1:172.21.15.70:6801/664757345] conn(0x7fceb40760b0 0x7fceb40784f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.683+0000 7fcec57fa640 1 -- 172.21.15.44:0/1198105874 --> [v2:172.21.15.70:6800/664757345,v1:172.21.15.70:6801/664757345] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fceb802eea0 con 0x7fceb40760b0 2024-04-22T06:31:36.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.683+0000 7fcec57fa640 1 -- 172.21.15.44:0/1198105874 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fceb8076c60 con 0x7fcec81500b0 2024-04-22T06:31:36.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.683+0000 7fc91e59c640 1 --2- 172.21.15.44:0/3788021894 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fc90006c9f0 0x7fc90006eeb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc918002230 tx=0x7fc918009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:36.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.683+0000 7fcec7fff640 1 --2- 172.21.15.44:0/1198105874 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fceb406c9c0 0x7fceb406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.683+0000 7f22bf7fe640 1 --2- 172.21.15.44:0/3432976022 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f22a806c8f0 0x7f22a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.683+0000 7f22bf7fe640 1 -- 172.21.15.44:0/3432976022 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f22b80ae7d0 con 0x7f22c806d470 2024-04-22T06:31:36.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.683+0000 7f8addc11640 1 --2- 172.21.15.44:0/4179939059 >> [v2:172.21.15.44:6800/2544087903,v1:172.21.15.44:6801/2544087903] conn(0x7f8abc076d40 0x7f8abc0791e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:36.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.683+0000 7fc90effd640 1 -- 172.21.15.44:0/3788021894 <== osd.3 v2:172.21.15.44:6824/1692928109 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc900078bf0 con 0x7fc9000760e0 2024-04-22T06:31:36.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.683+0000 7fa1af7fe640 1 --2- 172.21.15.44:0/2287425767 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fa19c06c8f0 0x7fa19c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.684+0000 7f314d9ef640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f314004ba90 0x7f31400e2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.684+0000 7f314d9ef640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31400dd3f0 0x7f314016f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.684+0000 7fcecd3f9640 1 --2- 172.21.15.44:0/1198105874 >> [v2:172.21.15.70:6800/664757345,v1:172.21.15.70:6801/664757345] conn(0x7fceb40760b0 0x7fceb40784f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.684+0000 7f314d9ef640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31400e37d0 0x7f31400479f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.684+0000 7f314d9ef640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f31400459b0 con 0x7f31400dd3f0 2024-04-22T06:31:36.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.684+0000 7f314d9ef640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f31400456d0 con 0x7f314004ba90 2024-04-22T06:31:36.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.684+0000 7f314d9ef640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3140045c90 con 0x7f31400e37d0 2024-04-22T06:31:36.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.684+0000 7f3146ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f314004ba90 0x7f31400e2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.683+0000 7f22bf7fe640 1 --2- 172.21.15.44:0/3432976022 >> [v2:172.21.15.70:6808/1551180300,v1:172.21.15.70:6809/1551180300] conn(0x7f22a8076070 0x7f22a80784b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.684+0000 7f22bf7fe640 1 -- 172.21.15.44:0/3432976022 --> [v2:172.21.15.70:6808/1551180300,v1:172.21.15.70:6809/1551180300] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f22b802ee50 con 0x7f22a8076070 2024-04-22T06:31:36.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.684+0000 7f22bf7fe640 1 -- 172.21.15.44:0/3432976022 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f22b80aebc0 con 0x7f22c806d470 2024-04-22T06:31:36.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.684+0000 7f22c6d9d640 1 --2- 172.21.15.44:0/3432976022 >> [v2:172.21.15.70:6808/1551180300,v1:172.21.15.70:6809/1551180300] conn(0x7f22a8076070 0x7f22a80784b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.684+0000 7fa197fff640 1 -- 172.21.15.44:0/2287425767 <== osd.0 v2:172.21.15.44:6816/2220268493 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa19c078b70 con 0x7fa19c076060 2024-04-22T06:31:36.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.684+0000 7fa1af7fe640 1 --2- 172.21.15.44:0/2287425767 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fa19c06c8f0 0x7fa19c06edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0006ca0 tx=0x7fa1a0009240 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:36.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.684+0000 7f3146ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f314004ba90 0x7f31400e2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:34130/0 (socket says 172.21.15.44:34130) 2024-04-22T06:31:36.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.684+0000 7f3146ffd640 1 -- 172.21.15.44:0/967218447 learned_addr learned my addr 172.21.15.44:0/967218447 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:36.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.684+0000 7f31467fc640 1 --2- 172.21.15.44:0/967218447 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31400dd3f0 0x7f314016f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.684+0000 7fcec7fff640 1 --2- 172.21.15.44:0/1198105874 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fceb406c9c0 0x7fceb406ee80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcec815a8c0 tx=0x7fcebc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:36.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.684+0000 7f22bdd9b640 1 --2- 172.21.15.44:0/3432976022 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f22a806c8f0 0x7f22a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.684+0000 7fcecd3f9640 1 --2- 172.21.15.44:0/1198105874 >> [v2:172.21.15.70:6800/664757345,v1:172.21.15.70:6801/664757345] conn(0x7fceb40760b0 0x7fceb40784f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:36.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.684+0000 7f8acdffb640 1 -- 172.21.15.44:0/4179939059 <== osd.2 v2:172.21.15.44:6800/2544087903 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8abc0798b0 con 0x7f8abc076d40 2024-04-22T06:31:36.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.684+0000 7f22c6d9d640 1 --2- 172.21.15.44:0/3432976022 >> [v2:172.21.15.70:6808/1551180300,v1:172.21.15.70:6809/1551180300] conn(0x7f22a8076070 0x7f22a80784b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:36.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.685+0000 7f3146ffd640 1 -- 172.21.15.44:0/967218447 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31400e37d0 msgr2=0x7f31400479f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.685+0000 7f31477fe640 1 --2- 172.21.15.44:0/967218447 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31400e37d0 0x7f31400479f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.685+0000 7f3146ffd640 1 --2- 172.21.15.44:0/967218447 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31400e37d0 0x7f31400479f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.685+0000 7f3146ffd640 1 -- 172.21.15.44:0/967218447 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31400dd3f0 msgr2=0x7f314016f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.685+0000 7f3146ffd640 1 --2- 172.21.15.44:0/967218447 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31400dd3f0 0x7f314016f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.685+0000 7f3146ffd640 1 -- 172.21.15.44:0/967218447 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3140048310 con 0x7f314004ba90 2024-04-22T06:31:36.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.685+0000 7f31467fc640 1 --2- 172.21.15.44:0/967218447 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31400dd3f0 0x7f314016f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:31:36.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.685+0000 7f31477fe640 1 --2- 172.21.15.44:0/967218447 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31400e37d0 0x7f31400479f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:31:36.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.685+0000 7f22bdd9b640 1 --2- 172.21.15.44:0/3432976022 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f22a806c8f0 0x7f22a806edb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f22c81f5360 tx=0x7f22b0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:36.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.685+0000 7fcec57fa640 1 -- 172.21.15.44:0/1198105874 <== osd.5 v2:172.21.15.70:6800/664757345 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fceb802eea0 con 0x7fceb40760b0 2024-04-22T06:31:36.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.685+0000 7f22bf7fe640 1 -- 172.21.15.44:0/3432976022 <== osd.7 v2:172.21.15.70:6808/1551180300 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f22b802ee50 con 0x7f22a8076070 2024-04-22T06:31:36.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.686+0000 7f3146ffd640 1 --2- 172.21.15.44:0/967218447 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f314004ba90 0x7f31400e2e00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f313802c050 tx=0x7f31380046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:36.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.688+0000 7f312ffff640 1 -- 172.21.15.44:0/967218447 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3138045030 con 0x7f314004ba90 2024-04-22T06:31:36.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.688+0000 7f312ffff640 1 -- 172.21.15.44:0/967218447 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31380359e0 con 0x7f314004ba90 2024-04-22T06:31:36.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.688+0000 7f312ffff640 1 -- 172.21.15.44:0/967218447 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3138035dd0 con 0x7f314004ba90 2024-04-22T06:31:36.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.688+0000 7f314d9ef640 1 -- 172.21.15.44:0/967218447 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3140179280 con 0x7f314004ba90 2024-04-22T06:31:36.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.688+0000 7f314d9ef640 1 -- 172.21.15.44:0/967218447 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3140179840 con 0x7f314004ba90 2024-04-22T06:31:36.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.689+0000 7f314d9ef640 1 -- 172.21.15.44:0/967218447 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3128001030 con 0x7f314004ba90 2024-04-22T06:31:36.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.691+0000 7f312ffff640 1 -- 172.21.15.44:0/967218447 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3138048070 con 0x7f314004ba90 2024-04-22T06:31:36.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.691+0000 7f312ffff640 1 --2- 172.21.15.44:0/967218447 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f313406c9c0 0x7f313406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.692+0000 7f312ffff640 1 -- 172.21.15.44:0/967218447 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f31380adba0 con 0x7f314004ba90 2024-04-22T06:31:36.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.692+0000 7f312ffff640 1 --2- 172.21.15.44:0/967218447 >> [v2:172.21.15.70:6816/1623253482,v1:172.21.15.70:6817/1623253482] conn(0x7f3134076130 0x7f3134078570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:36.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.692+0000 7f312ffff640 1 -- 172.21.15.44:0/967218447 --> [v2:172.21.15.70:6816/1623253482,v1:172.21.15.70:6817/1623253482] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3134078c40 con 0x7f3134076130 2024-04-22T06:31:36.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.692+0000 7f312ffff640 1 -- 172.21.15.44:0/967218447 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f31380adf90 con 0x7f314004ba90 2024-04-22T06:31:36.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.692+0000 7f31467fc640 1 --2- 172.21.15.44:0/967218447 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f313406c9c0 0x7f313406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.693+0000 7f31477fe640 1 --2- 172.21.15.44:0/967218447 >> [v2:172.21.15.70:6816/1623253482,v1:172.21.15.70:6817/1623253482] conn(0x7f3134076130 0x7f3134078570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:36.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.694+0000 7f31467fc640 1 --2- 172.21.15.44:0/967218447 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f313406c9c0 0x7f313406ee80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f31400069a0 tx=0x7f313c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:36.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.694+0000 7f31477fe640 1 --2- 172.21.15.44:0/967218447 >> [v2:172.21.15.70:6816/1623253482,v1:172.21.15.70:6817/1623253482] conn(0x7f3134076130 0x7f3134078570 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:36.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.695+0000 7f312ffff640 1 -- 172.21.15.44:0/967218447 <== osd.6 v2:172.21.15.70:6816/1623253482 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3134078c40 con 0x7f3134076130 2024-04-22T06:31:36.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.713+0000 7f3823e9a640 1 -- 172.21.15.44:0/1195231814 --> [v2:172.21.15.70:6818/3802577427,v1:172.21.15.70:6820/3802577427] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3804002dd0 con 0x7f3800075f60 2024-04-22T06:31:36.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.715+0000 7f38127fc640 1 -- 172.21.15.44:0/1195231814 <== osd.4 v2:172.21.15.70:6818/3802577427 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3804002dd0 con 0x7f3800075f60 2024-04-22T06:31:36.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.715+0000 7fc924aa8640 1 -- 172.21.15.44:0/3788021894 --> [v2:172.21.15.44:6824/1692928109,v1:172.21.15.44:6825/1692928109] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc9200c89d0 con 0x7fc9000760e0 2024-04-22T06:31:36.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.715+0000 7f37fffff640 1 -- 172.21.15.44:0/1195231814 >> [v2:172.21.15.70:6818/3802577427,v1:172.21.15.70:6820/3802577427] conn(0x7f3800075f60 msgr2=0x7f38000783a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.715+0000 7f37fffff640 1 --2- 172.21.15.44:0/1195231814 >> [v2:172.21.15.70:6818/3802577427,v1:172.21.15.70:6820/3802577427] conn(0x7f3800075f60 0x7f38000783a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.716+0000 7f37fffff640 1 -- 172.21.15.44:0/1195231814 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f380006c8f0 msgr2=0x7f380006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.716+0000 7f37fffff640 1 --2- 172.21.15.44:0/1195231814 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f380006c8f0 0x7f380006edb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f380c00a810 tx=0x7f380c008040 comp rx=0 tx=0).stop 2024-04-22T06:31:36.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.716+0000 7f37fffff640 1 -- 172.21.15.44:0/1195231814 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38140e6f70 msgr2=0x7f3814008770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.716+0000 7f37fffff640 1 --2- 172.21.15.44:0/1195231814 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38140e6f70 0x7f3814008770 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f381c000b60 tx=0x7f381c0aa070 comp rx=0 tx=0).stop 2024-04-22T06:31:36.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.716+0000 7f3822437640 1 -- 172.21.15.44:0/1195231814 reap_dead start 2024-04-22T06:31:36.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.716+0000 7fc90effd640 1 -- 172.21.15.44:0/3788021894 <== osd.3 v2:172.21.15.44:6824/1692928109 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc9200c89d0 con 0x7fc9000760e0 2024-04-22T06:31:36.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.716+0000 7f37fffff640 1 -- 172.21.15.44:0/1195231814 shutdown_connections 2024-04-22T06:31:36.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.716+0000 7f37fffff640 1 -- 172.21.15.44:0/1195231814 >> 172.21.15.44:0/1195231814 conn(0x7f381400c3a0 msgr2=0x7f381400c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:36.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.716+0000 7f37fffff640 1 -- 172.21.15.44:0/1195231814 shutdown_connections 2024-04-22T06:31:36.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.716+0000 7f37fffff640 1 -- 172.21.15.44:0/1195231814 wait complete. 2024-04-22T06:31:36.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.717+0000 7fc90cff9640 1 -- 172.21.15.44:0/3788021894 >> [v2:172.21.15.44:6824/1692928109,v1:172.21.15.44:6825/1692928109] conn(0x7fc9000760e0 msgr2=0x7fc900078520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.717+0000 7fc90cff9640 1 --2- 172.21.15.44:0/3788021894 >> [v2:172.21.15.44:6824/1692928109,v1:172.21.15.44:6825/1692928109] conn(0x7fc9000760e0 0x7fc900078520 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.717+0000 7fa1b4dbf640 1 -- 172.21.15.44:0/2287425767 --> [v2:172.21.15.44:6816/2220268493,v1:172.21.15.44:6817/2220268493] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa1b00761b0 con 0x7fa19c076060 2024-04-22T06:31:36.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.717+0000 7fa197fff640 1 -- 172.21.15.44:0/2287425767 <== osd.0 v2:172.21.15.44:6816/2220268493 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa1b00761b0 con 0x7fa19c076060 2024-04-22T06:31:36.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.718+0000 7fc90cff9640 1 -- 172.21.15.44:0/3788021894 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fc90006c9f0 msgr2=0x7fc90006eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.718+0000 7fc90cff9640 1 --2- 172.21.15.44:0/3788021894 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fc90006c9f0 0x7fc90006eeb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc918002230 tx=0x7fc918009040 comp rx=0 tx=0).stop 2024-04-22T06:31:36.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.718+0000 7fc90cff9640 1 -- 172.21.15.44:0/3788021894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9201595b0 msgr2=0x7fc9201577d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.718+0000 7fc90cff9640 1 --2- 172.21.15.44:0/3788021894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9201595b0 0x7fc9201577d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc91000c780 tx=0x7fc91000cc40 comp rx=0 tx=0).stop 2024-04-22T06:31:36.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.718+0000 7fa1b4dbf640 1 -- 172.21.15.44:0/2287425767 >> [v2:172.21.15.44:6816/2220268493,v1:172.21.15.44:6817/2220268493] conn(0x7fa19c076060 msgr2=0x7fa19c0784a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.718+0000 7fa1b4dbf640 1 --2- 172.21.15.44:0/2287425767 >> [v2:172.21.15.44:6816/2220268493,v1:172.21.15.44:6817/2220268493] conn(0x7fa19c076060 0x7fa19c0784a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.718+0000 7fc91ed9d640 1 -- 172.21.15.44:0/3788021894 reap_dead start 2024-04-22T06:31:36.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.718+0000 7fc90cff9640 1 -- 172.21.15.44:0/3788021894 shutdown_connections 2024-04-22T06:31:36.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.718+0000 7fc90cff9640 1 -- 172.21.15.44:0/3788021894 >> 172.21.15.44:0/3788021894 conn(0x7fc9200cd630 msgr2=0x7fc920070770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:36.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.718+0000 7fa1b4dbf640 1 -- 172.21.15.44:0/2287425767 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fa19c06c8f0 msgr2=0x7fa19c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.718+0000 7fa1b4dbf640 1 --2- 172.21.15.44:0/2287425767 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fa19c06c8f0 0x7fa19c06edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0006ca0 tx=0x7fa1a0009240 comp rx=0 tx=0).stop 2024-04-22T06:31:36.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.718+0000 7fa1b4dbf640 1 -- 172.21.15.44:0/2287425767 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa1b0151110 msgr2=0x7fa1b0141f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.718+0000 7fa1b4dbf640 1 --2- 172.21.15.44:0/2287425767 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa1b0151110 0x7fa1b0141f40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8009ee0 tx=0x7fa1a8004a40 comp rx=0 tx=0).stop 2024-04-22T06:31:36.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.718+0000 7f09f9873640 1 -- 172.21.15.44:0/1241774191 --> [v2:172.21.15.44:6808/1278678977,v1:172.21.15.44:6809/1278678977] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f09ec181570 con 0x7f09d8076360 2024-04-22T06:31:36.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.718+0000 7f09f17fa640 1 -- 172.21.15.44:0/1241774191 <== osd.1 v2:172.21.15.44:6808/1278678977 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f09ec181570 con 0x7f09d8076360 2024-04-22T06:31:36.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.718+0000 7fc90cff9640 1 -- 172.21.15.44:0/3788021894 shutdown_connections 2024-04-22T06:31:36.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.719+0000 7fa1affff640 1 -- 172.21.15.44:0/2287425767 reap_dead start 2024-04-22T06:31:36.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.719+0000 7fa1b4dbf640 1 -- 172.21.15.44:0/2287425767 shutdown_connections 2024-04-22T06:31:36.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.719+0000 7fc90cff9640 1 -- 172.21.15.44:0/3788021894 wait complete. 2024-04-22T06:31:36.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.719+0000 7fa1b4dbf640 1 -- 172.21.15.44:0/2287425767 >> 172.21.15.44:0/2287425767 conn(0x7fa1b00cc840 msgr2=0x7fa1b00c6f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:36.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.719+0000 7f09f9873640 1 -- 172.21.15.44:0/1241774191 >> [v2:172.21.15.44:6808/1278678977,v1:172.21.15.44:6809/1278678977] conn(0x7f09d8076360 msgr2=0x7f09d80787a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.719+0000 7f09f9873640 1 --2- 172.21.15.44:0/1241774191 >> [v2:172.21.15.44:6808/1278678977,v1:172.21.15.44:6809/1278678977] conn(0x7f09d8076360 0x7f09d80787a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.719+0000 7f09f9873640 1 -- 172.21.15.44:0/1241774191 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f09d806ccf0 msgr2=0x7f09d806f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.719+0000 7fa1b4dbf640 1 -- 172.21.15.44:0/2287425767 shutdown_connections 2024-04-22T06:31:36.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.719+0000 7fa1b4dbf640 1 -- 172.21.15.44:0/2287425767 wait complete. 2024-04-22T06:31:36.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.719+0000 7f8abb7fe640 1 -- 172.21.15.44:0/4179939059 --> [v2:172.21.15.44:6800/2544087903,v1:172.21.15.44:6801/2544087903] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8ad8069310 con 0x7f8abc076d40 2024-04-22T06:31:36.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.719+0000 7f09f9873640 1 --2- 172.21.15.44:0/1241774191 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f09d806ccf0 0x7f09d806f1b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f09e0036000 tx=0x7f09e0008030 comp rx=0 tx=0).stop 2024-04-22T06:31:36.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.719+0000 7f09f9873640 1 -- 172.21.15.44:0/1241774191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09ec005ee0 msgr2=0x7f09ec0d2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.719+0000 7f09f9873640 1 --2- 172.21.15.44:0/1241774191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09ec005ee0 0x7f09ec0d2680 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f09e800d800 tx=0x7f09e800dcc0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.719+0000 7f09f9072640 1 -- 172.21.15.44:0/1241774191 reap_dead start 2024-04-22T06:31:36.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.719+0000 7f8acdffb640 1 -- 172.21.15.44:0/4179939059 <== osd.2 v2:172.21.15.44:6800/2544087903 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8ad8069310 con 0x7f8abc076d40 2024-04-22T06:31:36.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.720+0000 7f09f9873640 1 -- 172.21.15.44:0/1241774191 shutdown_connections 2024-04-22T06:31:36.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.720+0000 7f09f9873640 1 -- 172.21.15.44:0/1241774191 >> 172.21.15.44:0/1241774191 conn(0x7f09ec050180 msgr2=0x7f09ec0d8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:36.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.720+0000 7f22ccd85640 1 -- 172.21.15.44:0/3432976022 --> [v2:172.21.15.70:6808/1551180300,v1:172.21.15.70:6809/1551180300] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f22c80cbcd0 con 0x7f22a8076070 2024-04-22T06:31:36.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.720+0000 7fcecee5c640 1 -- 172.21.15.44:0/1198105874 --> [v2:172.21.15.70:6800/664757345,v1:172.21.15.70:6801/664757345] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcea8002d50 con 0x7fceb40760b0 2024-04-22T06:31:36.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.720+0000 7f09f9873640 1 -- 172.21.15.44:0/1241774191 shutdown_connections 2024-04-22T06:31:36.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.720+0000 7f09f9873640 1 -- 172.21.15.44:0/1241774191 wait complete. 2024-04-22T06:31:36.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.720+0000 7f8abb7fe640 1 -- 172.21.15.44:0/4179939059 >> [v2:172.21.15.44:6800/2544087903,v1:172.21.15.44:6801/2544087903] conn(0x7f8abc076d40 msgr2=0x7f8abc0791e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.720+0000 7f8abb7fe640 1 --2- 172.21.15.44:0/4179939059 >> [v2:172.21.15.44:6800/2544087903,v1:172.21.15.44:6801/2544087903] conn(0x7f8abc076d40 0x7f8abc0791e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.720+0000 7f8abb7fe640 1 -- 172.21.15.44:0/4179939059 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f8abc06c970 msgr2=0x7f8abc06ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.721+0000 7f8abb7fe640 1 --2- 172.21.15.44:0/4179939059 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f8abc06c970 0x7f8abc06ee30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0035000 tx=0x7f8ad002e000 comp rx=0 tx=0).stop 2024-04-22T06:31:36.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.721+0000 7f8abb7fe640 1 -- 172.21.15.44:0/4179939059 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ad8070300 msgr2=0x7f8ad81458a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.721+0000 7f8abb7fe640 1 --2- 172.21.15.44:0/4179939059 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ad8070300 0x7f8ad81458a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400c410 tx=0x7f8ad400c8d0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.721+0000 7f8addc11640 1 -- 172.21.15.44:0/4179939059 reap_dead start 2024-04-22T06:31:36.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.721+0000 7fcec57fa640 1 -- 172.21.15.44:0/1198105874 <== osd.5 v2:172.21.15.70:6800/664757345 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fcea8002d50 con 0x7fceb40760b0 2024-04-22T06:31:36.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.720+0000 7f22bf7fe640 1 -- 172.21.15.44:0/3432976022 <== osd.7 v2:172.21.15.70:6808/1551180300 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f22c80cbcd0 con 0x7f22a8076070 2024-04-22T06:31:36.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.721+0000 7f8abb7fe640 1 -- 172.21.15.44:0/4179939059 shutdown_connections 2024-04-22T06:31:36.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.721+0000 7f8abb7fe640 1 -- 172.21.15.44:0/4179939059 >> 172.21.15.44:0/4179939059 conn(0x7f8ad80cd630 msgr2=0x7f8ad8073650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:36.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.721+0000 7f22ccd85640 1 -- 172.21.15.44:0/3432976022 >> [v2:172.21.15.70:6808/1551180300,v1:172.21.15.70:6809/1551180300] conn(0x7f22a8076070 msgr2=0x7f22a80784b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.721+0000 7f22ccd85640 1 --2- 172.21.15.44:0/3432976022 >> [v2:172.21.15.70:6808/1551180300,v1:172.21.15.70:6809/1551180300] conn(0x7f22a8076070 0x7f22a80784b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.721+0000 7f8abb7fe640 1 -- 172.21.15.44:0/4179939059 shutdown_connections 2024-04-22T06:31:36.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.721+0000 7f8abb7fe640 1 -- 172.21.15.44:0/4179939059 wait complete. 2024-04-22T06:31:36.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.721+0000 7fcecee5c640 1 -- 172.21.15.44:0/1198105874 >> [v2:172.21.15.70:6800/664757345,v1:172.21.15.70:6801/664757345] conn(0x7fceb40760b0 msgr2=0x7fceb40784f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.721+0000 7fcecee5c640 1 --2- 172.21.15.44:0/1198105874 >> [v2:172.21.15.70:6800/664757345,v1:172.21.15.70:6801/664757345] conn(0x7fceb40760b0 0x7fceb40784f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.721+0000 7fcecee5c640 1 -- 172.21.15.44:0/1198105874 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fceb406c9c0 msgr2=0x7fceb406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.721+0000 7fcecee5c640 1 --2- 172.21.15.44:0/1198105874 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fceb406c9c0 0x7fceb406ee80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcec815a8c0 tx=0x7fcebc009040 comp rx=0 tx=0).stop 2024-04-22T06:31:36.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.721+0000 7fcecee5c640 1 -- 172.21.15.44:0/1198105874 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcec81500b0 msgr2=0x7fcec8152b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.721+0000 7fcecee5c640 1 --2- 172.21.15.44:0/1198105874 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcec81500b0 0x7fcec8152b20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fceb8009a40 tx=0x7fceb8004730 comp rx=0 tx=0).stop 2024-04-22T06:31:36.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.721+0000 7fcecd3f9640 1 -- 172.21.15.44:0/1198105874 reap_dead start 2024-04-22T06:31:36.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.722+0000 7fcecee5c640 1 -- 172.21.15.44:0/1198105874 shutdown_connections 2024-04-22T06:31:36.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.722+0000 7fcecee5c640 1 -- 172.21.15.44:0/1198105874 >> 172.21.15.44:0/1198105874 conn(0x7fcec80750d0 msgr2=0x7fcec8158f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:36.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.722+0000 7fcecee5c640 1 -- 172.21.15.44:0/1198105874 shutdown_connections 2024-04-22T06:31:36.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.722+0000 7f22ccd85640 1 -- 172.21.15.44:0/3432976022 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f22a806c8f0 msgr2=0x7f22a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.722+0000 7f22ccd85640 1 --2- 172.21.15.44:0/3432976022 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f22a806c8f0 0x7f22a806edb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f22c81f5360 tx=0x7f22b0009040 comp rx=0 tx=0).stop 2024-04-22T06:31:36.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.722+0000 7f22ccd85640 1 -- 172.21.15.44:0/3432976022 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f22c806d470 msgr2=0x7f22c80cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.722+0000 7f22ccd85640 1 --2- 172.21.15.44:0/3432976022 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f22c806d470 0x7f22c80cd580 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f22b8000c00 tx=0x7f22b80046c0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.722+0000 7f22c6d9d640 1 -- 172.21.15.44:0/3432976022 reap_dead start 2024-04-22T06:31:36.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.722+0000 7fcecee5c640 1 -- 172.21.15.44:0/1198105874 wait complete. 2024-04-22T06:31:36.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.723+0000 7f22ccd85640 1 -- 172.21.15.44:0/3432976022 shutdown_connections 2024-04-22T06:31:36.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.723+0000 7f22ccd85640 1 -- 172.21.15.44:0/3432976022 >> 172.21.15.44:0/3432976022 conn(0x7f22c80733c0 msgr2=0x7f22c8153410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:36.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.723+0000 7f22ccd85640 1 -- 172.21.15.44:0/3432976022 shutdown_connections 2024-04-22T06:31:36.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.723+0000 7f22ccd85640 1 -- 172.21.15.44:0/3432976022 wait complete. 2024-04-22T06:31:36.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.726+0000 7f314d9ef640 1 -- 172.21.15.44:0/967218447 --> [v2:172.21.15.70:6816/1623253482,v1:172.21.15.70:6817/1623253482] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3128002dd0 con 0x7f3134076130 2024-04-22T06:31:36.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.727+0000 7f312ffff640 1 -- 172.21.15.44:0/967218447 <== osd.6 v2:172.21.15.70:6816/1623253482 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3128002dd0 con 0x7f3134076130 2024-04-22T06:31:36.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.728+0000 7f312dffb640 1 -- 172.21.15.44:0/967218447 >> [v2:172.21.15.70:6816/1623253482,v1:172.21.15.70:6817/1623253482] conn(0x7f3134076130 msgr2=0x7f3134078570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.728+0000 7f312dffb640 1 --2- 172.21.15.44:0/967218447 >> [v2:172.21.15.70:6816/1623253482,v1:172.21.15.70:6817/1623253482] conn(0x7f3134076130 0x7f3134078570 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.728+0000 7f312dffb640 1 -- 172.21.15.44:0/967218447 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f313406c9c0 msgr2=0x7f313406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.728+0000 7f312dffb640 1 --2- 172.21.15.44:0/967218447 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f313406c9c0 0x7f313406ee80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f31400069a0 tx=0x7f313c009040 comp rx=0 tx=0).stop 2024-04-22T06:31:36.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.728+0000 7f312dffb640 1 -- 172.21.15.44:0/967218447 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f314004ba90 msgr2=0x7f31400e2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:36.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.728+0000 7f312dffb640 1 --2- 172.21.15.44:0/967218447 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f314004ba90 0x7f31400e2e00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f313802c050 tx=0x7f31380046c0 comp rx=0 tx=0).stop 2024-04-22T06:31:36.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.728+0000 7f31477fe640 1 -- 172.21.15.44:0/967218447 reap_dead start 2024-04-22T06:31:36.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.729+0000 7f312dffb640 1 -- 172.21.15.44:0/967218447 shutdown_connections 2024-04-22T06:31:36.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.729+0000 7f312dffb640 1 -- 172.21.15.44:0/967218447 >> 172.21.15.44:0/967218447 conn(0x7f31400501b0 msgr2=0x7f3140006ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:36.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.732+0000 7f312dffb640 1 -- 172.21.15.44:0/967218447 shutdown_connections 2024-04-22T06:31:36.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.732+0000 7f312dffb640 1 -- 172.21.15.44:0/967218447 wait complete. 2024-04-22T06:31:36.782 INFO:teuthology.orchestra.run.smithi044.stdout:55834574860 2024-04-22T06:31:36.782 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-22T06:31:36.783 INFO:teuthology.orchestra.run.smithi044.stdout:55834574859 2024-04-22T06:31:36.783 INFO:teuthology.orchestra.run.smithi044.stdout:55834574859 2024-04-22T06:31:36.784 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-22T06:31:36.784 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-22T06:31:36.785 INFO:teuthology.orchestra.run.smithi044.stdout:55834574859 2024-04-22T06:31:36.785 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-22T06:31:36.786 INFO:teuthology.orchestra.run.smithi044.stdout:55834574858 2024-04-22T06:31:36.787 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-22T06:31:36.787 INFO:teuthology.orchestra.run.smithi044.stdout:55834574859 2024-04-22T06:31:36.788 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-22T06:31:36.788 INFO:teuthology.orchestra.run.smithi044.stdout:55834574859 2024-04-22T06:31:36.789 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-22T06:31:36.789 INFO:teuthology.orchestra.run.smithi044.stdout:55834574859 2024-04-22T06:31:36.790 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-22T06:31:36.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.998+0000 7f253f59e640 1 Processor -- start 2024-04-22T06:31:36.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.997+0000 7fb498746640 1 Processor -- start 2024-04-22T06:31:36.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.998+0000 7fb498746640 1 -- start start 2024-04-22T06:31:36.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.998+0000 7f253f59e640 1 -- start start 2024-04-22T06:31:36.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.999+0000 7f69155cc640 1 Processor -- start 2024-04-22T06:31:37.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.999+0000 7fb498746640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb4900732a0 0x7fb49015cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.999+0000 7fb498746640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb49015c3e0 con 0x7fb490153050 2024-04-22T06:31:37.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.999+0000 7fb4964e2640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fb490153050 0x7fb4900c8260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:49482/0 (socket says 172.21.15.44:49482) 2024-04-22T06:31:37.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.999+0000 7fb4964e2640 1 -- 172.21.15.44:0/1646229643 learned_addr learned my addr 172.21.15.44:0/1646229643 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:37.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.999+0000 7fb498746640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb490159fd0 con 0x7fb49015d1b0 2024-04-22T06:31:37.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.000+0000 7fb498746640 1 -- 172.21.15.44:0/1646229643 --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fb49015c7e0 con 0x7fb4900732a0 2024-04-22T06:31:37.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:36.999+0000 7f28fb22a640 1 Processor -- start 2024-04-22T06:31:37.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.000+0000 7f28fb22a640 1 -- start start 2024-04-22T06:31:37.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.000+0000 7f253f59e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f25380d9bf0 0x7f25380e3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.000+0000 7f253f59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2538046170 0x7f2538007ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.000+0000 7f253f59e640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2538046640 con 0x7f253804bc10 2024-04-22T06:31:37.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.000+0000 7f253f59e640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f25380081e0 con 0x7f25380d9bf0 2024-04-22T06:31:37.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.000+0000 7f253f59e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f25380d9fd0 con 0x7f2538046170 2024-04-22T06:31:37.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.000+0000 7f69155cc640 1 -- start start 2024-04-22T06:31:37.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.000+0000 7fb495ce1640 1 --2- 172.21.15.44:0/1646229643 >> v2:172.21.15.70:3300/0 conn(0x7fb4900732a0 0x7fb49015cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.000+0000 7fb4954e0640 1 -- 172.21.15.44:0/1646229643 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2914044711 0 0) 0x7fb49015c3e0 con 0x7fb490153050 2024-04-22T06:31:37.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.000+0000 7f253dd9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2538046170 0x7f2538007ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.000+0000 7efdd1093640 1 Processor -- start 2024-04-22T06:31:37.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.000+0000 7fb4954e0640 1 -- 172.21.15.44:0/1646229643 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb47c003520 con 0x7fb490153050 2024-04-22T06:31:37.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.000+0000 7efdd1093640 1 -- start start 2024-04-22T06:31:37.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.000+0000 7f253dd9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2538046170 0x7f2538007ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:34148/0 (socket says 172.21.15.44:34148) 2024-04-22T06:31:37.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.000+0000 7f253dd9b640 1 -- 172.21.15.44:0/134408625 learned_addr learned my addr 172.21.15.44:0/134408625 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:37.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.001+0000 7fb495ce1640 1 -- 172.21.15.44:0/1646229643 >> v1:172.21.15.70:6789/0 conn(0x7fb49015d1b0 legacy=0x7fb4901639b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.001+0000 7fb495ce1640 1 -- 172.21.15.44:0/1646229643 >> v1:172.21.15.44:6789/0 conn(0x7fb490153050 legacy=0x7fb4900c8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.001+0000 7fb4954e0640 1 -- 172.21.15.44:0/1646229643 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1530199805 0 0) 0x7fb47c003520 con 0x7fb490153050 2024-04-22T06:31:37.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.001+0000 7fb495ce1640 1 -- 172.21.15.44:0/1646229643 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb490159450 con 0x7fb4900732a0 2024-04-22T06:31:37.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.001+0000 7f253e59c640 1 --2- 172.21.15.44:0/134408625 >> v2:172.21.15.44:3300/0 conn(0x7f25380d9bf0 0x7f25380e3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.001+0000 7f28fb22a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f28f4151110 0x7f28f4157990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.001+0000 7f69155cc640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f69100c70a0 0x7f6910163670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.001+0000 7f28fa228640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f28f4151110 0x7f28f4157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.001+0000 7f69155cc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f69100b52c0 0x7f691015f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.001+0000 7f28fa228640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f28f4151110 0x7f28f4157990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:34154/0 (socket says 172.21.15.44:34154) 2024-04-22T06:31:37.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.001+0000 7f28fa228640 1 -- 172.21.15.44:0/3541061405 learned_addr learned my addr 172.21.15.44:0/3541061405 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:37.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.001+0000 7f28fb22a640 1 --2- 172.21.15.44:0/3541061405 >> v2:172.21.15.44:3300/0 conn(0x7f28f40c8600 0x7f28f40c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.001+0000 7f253d59a640 1 -- 172.21.15.44:0/134408625 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3110639457 0 0) 0x7f2538046640 con 0x7f253804bc10 2024-04-22T06:31:37.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.002+0000 7f28fb22a640 1 -- 172.21.15.44:0/3541061405 --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28f40c4740 con 0x7f28f40c4ca0 2024-04-22T06:31:37.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.002+0000 7f28fb22a640 1 -- 172.21.15.44:0/3541061405 --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f28f4074580 con 0x7f28f40c8600 2024-04-22T06:31:37.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.002+0000 7f28fb22a640 1 -- 172.21.15.44:0/3541061405 --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f28f40c4a20 con 0x7f28f4151110 2024-04-22T06:31:37.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.002+0000 7f253d59a640 1 -- 172.21.15.44:0/134408625 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2528003520 con 0x7f253804bc10 2024-04-22T06:31:37.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.002+0000 7f69155cc640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69100cca20 con 0x7f69100cba20 2024-04-22T06:31:37.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.002+0000 7f69155cc640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f69100cc7f0 con 0x7f69100b52c0 2024-04-22T06:31:37.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.002+0000 7f69155cc640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f691015fac0 con 0x7f69100c70a0 2024-04-22T06:31:37.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.001+0000 7fc494cfd640 1 Processor -- start 2024-04-22T06:31:37.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.002+0000 7efdd1093640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7efdcc153090 0x7efdcc070ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.002+0000 7efdd1093640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efdcc153470 0x7efdcc0c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.002+0000 7efdd1093640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdcc0c7690 con 0x7efdcc0c60c0 2024-04-22T06:31:37.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.002+0000 7efdd1093640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7efdcc1590b0 con 0x7efdcc153090 2024-04-22T06:31:37.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.002+0000 7efdd1093640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7efdcc159390 con 0x7efdcc153470 2024-04-22T06:31:37.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.002+0000 7f690f7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f69100b52c0 0x7f691015f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.002+0000 7f690f7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f69100b52c0 0x7f691015f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:34156/0 (socket says 172.21.15.44:34156) 2024-04-22T06:31:37.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.002+0000 7f690f7fe640 1 -- 172.21.15.44:0/3352389379 learned_addr learned my addr 172.21.15.44:0/3352389379 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:37.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.002+0000 7efdcad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7efdcc153090 0x7efdcc070ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.002+0000 7efdcad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7efdcc153090 0x7efdcc070ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:34158/0 (socket says 172.21.15.44:34158) 2024-04-22T06:31:37.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.002+0000 7efdcad9d640 1 -- 172.21.15.44:0/4275282871 learned_addr learned my addr 172.21.15.44:0/4275282871 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:37.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.001+0000 7fc494cfd640 1 -- start start 2024-04-22T06:31:37.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.002+0000 7efdca59c640 1 --2- 172.21.15.44:0/4275282871 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efdcc153470 0x7efdcc0c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.002+0000 7fecced92640 1 Processor -- start 2024-04-22T06:31:37.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.002+0000 7f690ffff640 1 --2- 172.21.15.44:0/3352389379 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f69100c70a0 0x7f6910163670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.002+0000 7f28fa228640 1 -- 172.21.15.44:0/3541061405 >> v2:172.21.15.44:3300/0 conn(0x7f28f40c8600 msgr2=0x7f28f40c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:37.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.002+0000 7f28fa228640 1 --2- 172.21.15.44:0/3541061405 >> v2:172.21.15.44:3300/0 conn(0x7f28f40c8600 0x7f28f40c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.002+0000 7f28fa228640 1 -- 172.21.15.44:0/3541061405 >> v1:172.21.15.44:6789/0 conn(0x7f28f40c4ca0 legacy=0x7f28f40c3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:37.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.003+0000 7f28fa228640 1 -- 172.21.15.44:0/3541061405 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f406ec90 con 0x7f28f4151110 2024-04-22T06:31:37.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.003+0000 7f253d59a640 1 -- 172.21.15.44:0/134408625 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1446747678 0 0) 0x7f2528003520 con 0x7f253804bc10 2024-04-22T06:31:37.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.003+0000 7fb495ce1640 1 --2- 172.21.15.44:0/1646229643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4900732a0 0x7fb49015cc70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb4800097a0 tx=0x7fb48002b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbf877867b0034f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:37.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.003+0000 7f253d59a640 1 -- 172.21.15.44:0/134408625 >> v2:172.21.15.70:3300/0 conn(0x7f2538046170 msgr2=0x7f2538007ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:37.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.003+0000 7f253d59a640 1 --2- 172.21.15.44:0/134408625 >> v2:172.21.15.70:3300/0 conn(0x7f2538046170 0x7f2538007ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.003+0000 7f253d59a640 1 -- 172.21.15.44:0/134408625 >> v2:172.21.15.44:3300/0 conn(0x7f25380d9bf0 msgr2=0x7f25380e3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:37.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.003+0000 7f253d59a640 1 --2- 172.21.15.44:0/134408625 >> v2:172.21.15.44:3300/0 conn(0x7f25380d9bf0 0x7f25380e3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.002+0000 7fecced92640 1 -- start start 2024-04-22T06:31:37.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.003+0000 7fc494cfd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc490152000 0x7fc49015bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.003+0000 7fc494cfd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc4900c5de0 0x7fc4900cab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.003+0000 7f253d59a640 1 -- 172.21.15.44:0/134408625 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2538006ee0 con 0x7f253804bc10 2024-04-22T06:31:37.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.003+0000 7fc494cfd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4900c88a0 0x7fc4900cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.003+0000 7fc48e59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc490152000 0x7fc49015bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.003+0000 7fc48e59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc490152000 0x7fc49015bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:34168/0 (socket says 172.21.15.44:34168) 2024-04-22T06:31:37.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.003+0000 7f253d59a640 1 -- 172.21.15.44:0/134408625 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1237339257 0 0) 0x7f2530001670 con 0x7f253804bc10 2024-04-22T06:31:37.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.003+0000 7fb4954e0640 1 -- 172.21.15.44:0/1646229643 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb480038070 con 0x7fb4900732a0 2024-04-22T06:31:37.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.003+0000 7fc48e59c640 1 -- 172.21.15.44:0/4063689652 learned_addr learned my addr 172.21.15.44:0/4063689652 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:37.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.003+0000 7fc494cfd640 1 -- 172.21.15.44:0/4063689652 --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fc4900c96a0 con 0x7fc4900c5de0 2024-04-22T06:31:37.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.003+0000 7fc494cfd640 1 -- 172.21.15.44:0/4063689652 --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fc49015b370 con 0x7fc490152000 2024-04-22T06:31:37.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.003+0000 7fc494cfd640 1 -- 172.21.15.44:0/4063689652 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc49015b930 con 0x7fc4900c88a0 2024-04-22T06:31:37.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.003+0000 7fc48ed9d640 1 --2- 172.21.15.44:0/4063689652 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4900c88a0 0x7fc4900cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.003+0000 7fc48dd9b640 1 --2- 172.21.15.44:0/4063689652 >> v2:172.21.15.44:3300/0 conn(0x7fc4900c5de0 0x7fc4900cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.003+0000 7f253d59a640 1 -- 172.21.15.44:0/134408625 >> v1:172.21.15.44:6789/0 conn(0x7f253804bc10 legacy=0x7f2538046e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.003+0000 7f253d59a640 1 --2- 172.21.15.44:0/134408625 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25280039b0 0x7f2528003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.003+0000 7f253d59a640 1 --2- 172.21.15.44:0/134408625 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2528004800 0x7f2528004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.003+0000 7fb4954e0640 1 -- 172.21.15.44:0/1646229643 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb480031500 con 0x7fb4900732a0 2024-04-22T06:31:37.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.003+0000 7fb4954e0640 1 -- 172.21.15.44:0/1646229643 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c001670 con 0x7fb4900732a0 2024-04-22T06:31:37.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.003+0000 7f253d59a640 1 --2- 172.21.15.44:0/134408625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25280051c0 0x7f252800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.004+0000 7f253d59a640 1 -- 172.21.15.44:0/134408625 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f25380081e0 con 0x7f25280051c0 2024-04-22T06:31:37.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.004+0000 7f253d59a640 1 -- 172.21.15.44:0/134408625 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f25380d9fd0 con 0x7f25280039b0 2024-04-22T06:31:37.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.004+0000 7f253d59a640 1 -- 172.21.15.44:0/134408625 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f252800bf30 con 0x7f2528004800 2024-04-22T06:31:37.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.004+0000 7fecced92640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fecc00dd4d0 0x7fecc0045560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.004+0000 7fecced92640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fecc0046510 0x7fecc0046960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.004+0000 7fecced92640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecc00dd8b0 con 0x7fecc0045aa0 2024-04-22T06:31:37.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.004+0000 7fecced92640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fecc00e4050 con 0x7fecc0046510 2024-04-22T06:31:37.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.004+0000 7fecced92640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fecc0046ea0 con 0x7fecc00dd4d0 2024-04-22T06:31:37.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.004+0000 7f690f7fe640 1 -- 172.21.15.44:0/3352389379 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f69100c70a0 msgr2=0x7f6910163670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:37.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.003+0000 7efdcad9d640 1 -- 172.21.15.44:0/4275282871 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efdcc153470 msgr2=0x7efdcc0c5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:37.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.004+0000 7efdc9d9b640 1 -- 172.21.15.44:0/4275282871 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1871217145 0 0) 0x7efdcc0c7690 con 0x7efdcc0c60c0 2024-04-22T06:31:37.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.004+0000 7fc48dd9b640 1 -- 172.21.15.44:0/4063689652 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4900c88a0 msgr2=0x7fc4900cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:37.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.004+0000 7fc48dd9b640 1 --2- 172.21.15.44:0/4063689652 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4900c88a0 0x7fc4900cd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.004+0000 7fc48dd9b640 1 -- 172.21.15.44:0/4063689652 >> v2:172.21.15.70:3300/0 conn(0x7fc490152000 msgr2=0x7fc49015bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:37.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.004+0000 7fc48dd9b640 1 --2- 172.21.15.44:0/4063689652 >> v2:172.21.15.70:3300/0 conn(0x7fc490152000 0x7fc49015bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.004+0000 7fc48dd9b640 1 -- 172.21.15.44:0/4063689652 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4900cb090 con 0x7fc4900c5de0 2024-04-22T06:31:37.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.004+0000 7f253e59c640 1 --2- 172.21.15.44:0/134408625 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25280039b0 0x7f2528003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.004+0000 7f690f7fe640 1 --2- 172.21.15.44:0/3352389379 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f69100c70a0 0x7f6910163670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.004+0000 7f690f7fe640 1 -- 172.21.15.44:0/3352389379 >> v1:172.21.15.44:6789/0 conn(0x7f69100cba20 legacy=0x7f69100d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.004+0000 7f690f7fe640 1 -- 172.21.15.44:0/3352389379 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69100ccfa0 con 0x7f69100b52c0 2024-04-22T06:31:37.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.004+0000 7f253dd9b640 1 --2- 172.21.15.44:0/134408625 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2528004800 0x7f2528004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.004+0000 7feccdd90640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fecc00dd4d0 0x7fecc0045560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.004+0000 7f690ffff640 1 --2- 172.21.15.44:0/3352389379 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f69100c70a0 0x7f6910163670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:31:37.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.004+0000 7fecce591640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fecc0046510 0x7fecc0046960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.004+0000 7fecce591640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fecc0046510 0x7fecc0046960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:34186/0 (socket says 172.21.15.44:34186) 2024-04-22T06:31:37.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.004+0000 7f253ed9d640 1 --2- 172.21.15.44:0/134408625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25280051c0 0x7f252800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.004+0000 7fecce591640 1 -- 172.21.15.44:0/2970912135 learned_addr learned my addr 172.21.15.44:0/2970912135 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:37.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.004+0000 7fb498746640 1 -- 172.21.15.44:0/1646229643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4900732a0 msgr2=0x7fb49015cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.004+0000 7fb498746640 1 --2- 172.21.15.44:0/1646229643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4900732a0 0x7fb49015cc70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb4800097a0 tx=0x7fb48002b820 comp rx=0 tx=0).stop 2024-04-22T06:31:37.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.004+0000 7f253dd9b640 1 -- 172.21.15.44:0/134408625 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25280039b0 msgr2=0x7f2528003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:37.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.004+0000 7f253dd9b640 1 --2- 172.21.15.44:0/134408625 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25280039b0 0x7f2528003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.004+0000 7f253dd9b640 1 -- 172.21.15.44:0/134408625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25280051c0 msgr2=0x7f252800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:37.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.005+0000 7f28fa228640 1 --2- 172.21.15.44:0/3541061405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28f4151110 0x7f28f4157990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f28f0007360 tx=0x7f28f002cac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=345aaa6e076c46cc server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:37.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.005+0000 7fb498746640 1 -- 172.21.15.44:0/1646229643 shutdown_connections 2024-04-22T06:31:37.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.005+0000 7f253dd9b640 1 --2- 172.21.15.44:0/134408625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25280051c0 0x7f252800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.005+0000 7f253dd9b640 1 -- 172.21.15.44:0/134408625 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2538006ee0 con 0x7f2528004800 2024-04-22T06:31:37.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.005+0000 7f253ed9d640 1 --2- 172.21.15.44:0/134408625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25280051c0 0x7f252800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:31:37.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.005+0000 7f253ed9d640 1 -- 172.21.15.44:0/134408625 reap_dead start 2024-04-22T06:31:37.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.005+0000 7f253e59c640 1 --2- 172.21.15.44:0/134408625 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25280039b0 0x7f2528003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:31:37.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.005+0000 7fecce591640 1 -- 172.21.15.44:0/2970912135 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fecc00dd4d0 msgr2=0x7fecc0045560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:37.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.005+0000 7fecce591640 1 --2- 172.21.15.44:0/2970912135 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fecc00dd4d0 0x7fecc0045560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.005+0000 7fecce591640 1 -- 172.21.15.44:0/2970912135 >> v1:172.21.15.44:6789/0 conn(0x7fecc0045aa0 legacy=0x7fecc004bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:37.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.005+0000 7fb498746640 1 --2- 172.21.15.44:0/1646229643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4900732a0 0x7fb49015cc70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.005+0000 7fb498746640 1 -- 172.21.15.44:0/1646229643 >> 172.21.15.44:0/1646229643 conn(0x7fb490074cb0 msgr2=0x7fb490075090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:37.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.005+0000 7fecce591640 1 -- 172.21.15.44:0/2970912135 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc00e34d0 con 0x7fecc0046510 2024-04-22T06:31:37.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.005+0000 7f28f9226640 1 -- 172.21.15.44:0/3541061405 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f0008df0 con 0x7f28f4151110 2024-04-22T06:31:37.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.005+0000 7fc48dd9b640 1 --2- 172.21.15.44:0/4063689652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4900c5de0 0x7fc4900cab50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc49006c700 tx=0x7fc48402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3025c2f33dad67ed server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:37.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.005+0000 7fb498746640 1 -- 172.21.15.44:0/1646229643 shutdown_connections 2024-04-22T06:31:37.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.005+0000 7fb498746640 1 -- 172.21.15.44:0/1646229643 wait complete. 2024-04-22T06:31:37.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.005+0000 7f28f9226640 1 -- 172.21.15.44:0/3541061405 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28f000a0d0 con 0x7f28f4151110 2024-04-22T06:31:37.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.005+0000 7f28f9226640 1 -- 172.21.15.44:0/3541061405 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f000a4a0 con 0x7f28f4151110 2024-04-22T06:31:37.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.005+0000 7efdcad9d640 1 --2- 172.21.15.44:0/4275282871 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efdcc153470 0x7efdcc0c5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.005+0000 7efdcad9d640 1 -- 172.21.15.44:0/4275282871 >> v1:172.21.15.44:6789/0 conn(0x7efdcc0c60c0 legacy=0x7efdcc15c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.005+0000 7efdcad9d640 1 -- 172.21.15.44:0/4275282871 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdcc159670 con 0x7efdcc153090 2024-04-22T06:31:37.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.005+0000 7ff7d5b30640 1 Processor -- start 2024-04-22T06:31:37.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.006+0000 7f253dd9b640 1 --2- 172.21.15.44:0/134408625 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2528004800 0x7f2528004c80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2540064fa0 tx=0x7f2528005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7e8486c263790c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:37.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.006+0000 7fecce591640 1 --2- 172.21.15.44:0/2970912135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecc0046510 0x7fecc0046960 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fecb8009870 tx=0x7fecb802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f52383be08d853 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:37.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.005+0000 7fc48d59a640 1 -- 172.21.15.44:0/4063689652 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc484035020 con 0x7fc4900c5de0 2024-04-22T06:31:37.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.005+0000 7fc48d59a640 1 -- 172.21.15.44:0/4063689652 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc484031830 con 0x7fc4900c5de0 2024-04-22T06:31:37.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.005+0000 7fc48d59a640 1 -- 172.21.15.44:0/4063689652 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc484031c00 con 0x7fc4900c5de0 2024-04-22T06:31:37.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.006+0000 7f28fb22a640 1 -- 172.21.15.44:0/3541061405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28f4151110 msgr2=0x7f28f4157990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.006+0000 7f28fb22a640 1 --2- 172.21.15.44:0/3541061405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28f4151110 0x7f28f4157990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f28f0007360 tx=0x7f28f002cac0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.006+0000 7ff7d5b30640 1 -- start start 2024-04-22T06:31:37.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.006+0000 7f253d59a640 1 -- 172.21.15.44:0/134408625 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25400b5070 con 0x7f2528004800 2024-04-22T06:31:37.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.006+0000 7efdca59c640 1 --2- 172.21.15.44:0/4275282871 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efdcc153470 0x7efdcc0c5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:31:37.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.006+0000 7f28fb22a640 1 -- 172.21.15.44:0/3541061405 shutdown_connections 2024-04-22T06:31:37.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.006+0000 7f253d59a640 1 -- 172.21.15.44:0/134408625 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25400a8b50 con 0x7f2528004800 2024-04-22T06:31:37.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.006+0000 7f28fb22a640 1 --2- 172.21.15.44:0/3541061405 >> v2:172.21.15.44:3300/0 conn(0x7f28f40c8600 0x7f28f40c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.006+0000 7f28fb22a640 1 --2- 172.21.15.44:0/3541061405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28f4151110 0x7f28f4157990 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f28f0007360 tx=0x7f28f002cac0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.006+0000 7f28fb22a640 1 -- 172.21.15.44:0/3541061405 >> 172.21.15.44:0/3541061405 conn(0x7f28f40cc840 msgr2=0x7f28f40ccc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:37.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.006+0000 7efdcad9d640 1 --2- 172.21.15.44:0/4275282871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdcc153090 0x7efdcc070ec0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efdc4009e50 tx=0x7efdc402bfa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd3a29b1b62996de server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:37.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.006+0000 7fc494cfd640 1 -- 172.21.15.44:0/4063689652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4900c5de0 msgr2=0x7fc4900cab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.006+0000 7fc494cfd640 1 --2- 172.21.15.44:0/4063689652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4900c5de0 0x7fc4900cab50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc49006c700 tx=0x7fc48402b8d0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.006+0000 7f253d59a640 1 -- 172.21.15.44:0/134408625 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2540050250 con 0x7f2528004800 2024-04-22T06:31:37.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.006+0000 7f690f7fe640 1 --2- 172.21.15.44:0/3352389379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69100b52c0 0x7f691015f580 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6900009870 tx=0x7f690002b7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36e4a35fddb2155b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:37.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.007+0000 7efdc9d9b640 1 -- 172.21.15.44:0/4275282871 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc402cb40 con 0x7efdcc153090 2024-04-22T06:31:37.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.007+0000 7f28fb22a640 1 -- 172.21.15.44:0/3541061405 shutdown_connections 2024-04-22T06:31:37.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.007+0000 7f28fb22a640 1 -- 172.21.15.44:0/3541061405 wait complete. 2024-04-22T06:31:37.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.006+0000 7fecccd8e640 1 -- 172.21.15.44:0/2970912135 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb8034070 con 0x7fecc0046510 2024-04-22T06:31:37.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.007+0000 7f690effd640 1 -- 172.21.15.44:0/3352389379 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6900037070 con 0x7f69100b52c0 2024-04-22T06:31:37.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.007+0000 7f690effd640 1 -- 172.21.15.44:0/3352389379 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6900030710 con 0x7f69100b52c0 2024-04-22T06:31:37.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.007+0000 7f690effd640 1 -- 172.21.15.44:0/3352389379 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6900030ae0 con 0x7f69100b52c0 2024-04-22T06:31:37.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.007+0000 7efdc9d9b640 1 -- 172.21.15.44:0/4275282871 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdc402cce0 con 0x7efdcc153090 2024-04-22T06:31:37.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.007+0000 7fc494cfd640 1 -- 172.21.15.44:0/4063689652 shutdown_connections 2024-04-22T06:31:37.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.007+0000 7fc494cfd640 1 --2- 172.21.15.44:0/4063689652 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4900c88a0 0x7fc4900cd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.007+0000 7fc494cfd640 1 --2- 172.21.15.44:0/4063689652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4900c5de0 0x7fc4900cab50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.007+0000 7fc494cfd640 1 --2- 172.21.15.44:0/4063689652 >> v2:172.21.15.70:3300/0 conn(0x7fc490152000 0x7fc49015bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.007+0000 7fc494cfd640 1 -- 172.21.15.44:0/4063689652 >> 172.21.15.44:0/4063689652 conn(0x7fc490068100 msgr2=0x7fc49006b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:37.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.007+0000 7f253f59e640 1 -- 172.21.15.44:0/134408625 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2528004800 msgr2=0x7f2528004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.007+0000 7f253f59e640 1 --2- 172.21.15.44:0/134408625 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2528004800 0x7f2528004c80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2540064fa0 tx=0x7f2528005830 comp rx=0 tx=0).stop 2024-04-22T06:31:37.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.007+0000 7fecccd8e640 1 -- 172.21.15.44:0/2970912135 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecb8030840 con 0x7fecc0046510 2024-04-22T06:31:37.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.007+0000 7fecccd8e640 1 -- 172.21.15.44:0/2970912135 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb8030c10 con 0x7fecc0046510 2024-04-22T06:31:37.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.007+0000 7fb498746640 1 Processor -- start 2024-04-22T06:31:37.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.007+0000 7fb498746640 1 -- start start 2024-04-22T06:31:37.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.007+0000 7ff7d5b30640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff7d0151de0 0x7ff7d00cde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.007+0000 7ff7d5b30640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ff7d00cc580 0x7ff7d00cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.007+0000 7ff7d5b30640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7d00ccf10 con 0x7ff7d015b9d0 2024-04-22T06:31:37.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.007+0000 7ff7d5b30640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7ff7d00cc0f0 con 0x7ff7d00cc580 2024-04-22T06:31:37.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.007+0000 7ff7d5b30640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7ff7d015be10 con 0x7ff7d0151de0 2024-04-22T06:31:37.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.007+0000 7ff7d4b2e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff7d0151de0 0x7ff7d00cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.007+0000 7ff7d4b2e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff7d0151de0 0x7ff7d00cde00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:34188/0 (socket says 172.21.15.44:34188) 2024-04-22T06:31:37.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.007+0000 7ff7d4b2e640 1 -- 172.21.15.44:0/3093930472 learned_addr learned my addr 172.21.15.44:0/3093930472 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:37.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.007+0000 7ff7d532f640 1 --2- 172.21.15.44:0/3093930472 >> v2:172.21.15.44:3300/0 conn(0x7ff7d00cc580 0x7ff7d00cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.007+0000 7efdc9d9b640 1 -- 172.21.15.44:0/4275282871 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc40345f0 con 0x7efdcc153090 2024-04-22T06:31:37.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.007+0000 7f253f59e640 1 -- 172.21.15.44:0/134408625 shutdown_connections 2024-04-22T06:31:37.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.007+0000 7f253f59e640 1 --2- 172.21.15.44:0/134408625 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2528004800 0x7f2528004c80 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.008+0000 7f253f59e640 1 -- 172.21.15.44:0/134408625 >> 172.21.15.44:0/134408625 conn(0x7f2538051290 msgr2=0x7f25380516a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:37.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.008+0000 7ff7d532f640 1 -- 172.21.15.44:0/3093930472 >> v2:172.21.15.70:3300/0 conn(0x7ff7d0151de0 msgr2=0x7ff7d00cde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:37.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.008+0000 7f253f59e640 1 -- 172.21.15.44:0/134408625 shutdown_connections 2024-04-22T06:31:37.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.008+0000 7efdd1093640 1 -- 172.21.15.44:0/4275282871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdcc153090 msgr2=0x7efdcc070ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.008+0000 7efdd1093640 1 --2- 172.21.15.44:0/4275282871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdcc153090 0x7efdcc070ec0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efdc4009e50 tx=0x7efdc402bfa0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.008+0000 7f69155cc640 1 -- 172.21.15.44:0/3352389379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69100b52c0 msgr2=0x7f691015f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.008+0000 7ff7d532f640 1 --2- 172.21.15.44:0/3093930472 >> v2:172.21.15.70:3300/0 conn(0x7ff7d0151de0 0x7ff7d00cde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.008+0000 7f69155cc640 1 --2- 172.21.15.44:0/3352389379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69100b52c0 0x7f691015f580 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6900009870 tx=0x7f690002b7d0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.008+0000 7efdd1093640 1 -- 172.21.15.44:0/4275282871 shutdown_connections 2024-04-22T06:31:37.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.008+0000 7efdd1093640 1 --2- 172.21.15.44:0/4275282871 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efdcc153470 0x7efdcc0c5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.008+0000 7efdd1093640 1 --2- 172.21.15.44:0/4275282871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdcc153090 0x7efdcc070ec0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.008+0000 7efdd1093640 1 -- 172.21.15.44:0/4275282871 >> 172.21.15.44:0/4275282871 conn(0x7efdcc074cb0 msgr2=0x7efdcc0750c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:37.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.008+0000 7ff7d532f640 1 -- 172.21.15.44:0/3093930472 >> v1:172.21.15.70:6789/0 conn(0x7ff7d015b9d0 legacy=0x7ff7d006c9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-22T06:31:37.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.008+0000 7efdd1093640 1 -- 172.21.15.44:0/4275282871 shutdown_connections 2024-04-22T06:31:37.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.008+0000 7f69155cc640 1 -- 172.21.15.44:0/3352389379 shutdown_connections 2024-04-22T06:31:37.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.008+0000 7f69155cc640 1 --2- 172.21.15.44:0/3352389379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69100b52c0 0x7f691015f580 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.008+0000 7f69155cc640 1 --2- 172.21.15.44:0/3352389379 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f69100c70a0 0x7f6910163670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.008+0000 7f69155cc640 1 -- 172.21.15.44:0/3352389379 >> 172.21.15.44:0/3352389379 conn(0x7f6910075720 msgr2=0x7f6910072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:37.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.008+0000 7f69155cc640 1 -- 172.21.15.44:0/3352389379 shutdown_connections 2024-04-22T06:31:37.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.008+0000 7f69155cc640 1 -- 172.21.15.44:0/3352389379 wait complete. 2024-04-22T06:31:37.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.009+0000 7f28fb22a640 1 Processor -- start 2024-04-22T06:31:37.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.009+0000 7f28fb22a640 1 -- start start 2024-04-22T06:31:37.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.008+0000 7efdd1093640 1 -- 172.21.15.44:0/4275282871 wait complete. 2024-04-22T06:31:37.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.008+0000 7fc494cfd640 1 -- 172.21.15.44:0/4063689652 shutdown_connections 2024-04-22T06:31:37.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.009+0000 7fc494cfd640 1 -- 172.21.15.44:0/4063689652 wait complete. 2024-04-22T06:31:37.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.008+0000 7f253f59e640 1 -- 172.21.15.44:0/134408625 wait complete. 2024-04-22T06:31:37.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.008+0000 7fecced92640 1 -- 172.21.15.44:0/2970912135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecc0046510 msgr2=0x7fecc0046960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.009+0000 7fecced92640 1 --2- 172.21.15.44:0/2970912135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecc0046510 0x7fecc0046960 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fecb8009870 tx=0x7fecb802b980 comp rx=0 tx=0).stop 2024-04-22T06:31:37.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.009+0000 7fecced92640 1 -- 172.21.15.44:0/2970912135 shutdown_connections 2024-04-22T06:31:37.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.009+0000 7fecced92640 1 --2- 172.21.15.44:0/2970912135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecc0046510 0x7fecc0046960 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.009+0000 7fecced92640 1 --2- 172.21.15.44:0/2970912135 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fecc00dd4d0 0x7fecc0045560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.009+0000 7fecced92640 1 -- 172.21.15.44:0/2970912135 >> 172.21.15.44:0/2970912135 conn(0x7fecc00505c0 msgr2=0x7fecc00509d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:37.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.009+0000 7fb498746640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4900732a0 0x7fb4900cd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.009+0000 7fb498746640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb490153050 0x7fb4901538d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.009+0000 7fb498746640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb49015d1b0 0x7fb4901ef4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.009+0000 7fb498746640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb490159cf0 con 0x7fb4900732a0 2024-04-22T06:31:37.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.009+0000 7fb498746640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb490159a10 con 0x7fb49015d1b0 2024-04-22T06:31:37.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.009+0000 7fb498746640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb490153430 con 0x7fb490153050 2024-04-22T06:31:37.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.009+0000 7fb495ce1640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb490153050 0x7fb4901538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.008+0000 7ff7d532f640 1 -- 172.21.15.44:0/3093930472 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d00d1620 con 0x7ff7d00cc580 2024-04-22T06:31:37.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.008+0000 7ff7d4b2e640 1 --2- 172.21.15.44:0/3093930472 >> v2:172.21.15.70:3300/0 conn(0x7ff7d0151de0 0x7ff7d00cde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:31:37.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.009+0000 7fb495ce1640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb490153050 0x7fb4901538d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.44:45576/0 (socket says 172.21.15.44:45576) 2024-04-22T06:31:37.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.009+0000 7fb495ce1640 1 -- 172.21.15.44:0/1157952314 learned_addr learned my addr 172.21.15.44:0/1157952314 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:37.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.009+0000 7fb496ce3640 1 --2- 172.21.15.44:0/1157952314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb49015d1b0 0x7fb4901ef4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.009+0000 7fb4964e2640 1 --2- 172.21.15.44:0/1157952314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4900732a0 0x7fb4900cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.009+0000 7fb495ce1640 1 -- 172.21.15.44:0/1157952314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb49015d1b0 msgr2=0x7fb4901ef4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.009+0000 7ff7d532f640 1 --2- 172.21.15.44:0/3093930472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7d00cc580 0x7ff7d00cc9d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff7d00cd8b0 tx=0x7ff7c802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7999437113d216a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:37.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.010+0000 7fb495ce1640 1 --2- 172.21.15.44:0/1157952314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb49015d1b0 0x7fb4901ef4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.010+0000 7fb495ce1640 1 -- 172.21.15.44:0/1157952314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4900732a0 msgr2=0x7fb4900cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.010+0000 7fecced92640 1 -- 172.21.15.44:0/2970912135 shutdown_connections 2024-04-22T06:31:37.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.010+0000 7fb495ce1640 1 --2- 172.21.15.44:0/1157952314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4900732a0 0x7fb4900cd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.010+0000 7fb495ce1640 1 -- 172.21.15.44:0/1157952314 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4900cead0 con 0x7fb490153050 2024-04-22T06:31:37.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.010+0000 7fb495ce1640 1 --2- 172.21.15.44:0/1157952314 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb490153050 0x7fb4901538d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb48002be30 tx=0x7fb4800046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:37.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.010+0000 7ff7cf7fe640 1 -- 172.21.15.44:0/3093930472 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8039070 con 0x7ff7d00cc580 2024-04-22T06:31:37.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.010+0000 7ff7cf7fe640 1 -- 172.21.15.44:0/3093930472 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7c802cc70 con 0x7ff7d00cc580 2024-04-22T06:31:37.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.010+0000 7ff7cf7fe640 1 -- 172.21.15.44:0/3093930472 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8039070 con 0x7ff7d00cc580 2024-04-22T06:31:37.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.010+0000 7fecced92640 1 -- 172.21.15.44:0/2970912135 wait complete. 2024-04-22T06:31:37.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.010+0000 7f28fb22a640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f28f40c4ca0 0x7f28f41f1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.010+0000 7fb486ffd640 1 -- 172.21.15.44:0/1157952314 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb480043070 con 0x7fb490153050 2024-04-22T06:31:37.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.010+0000 7f28fb22a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28f40c8600 0x7f28f414f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.010+0000 7fb486ffd640 1 -- 172.21.15.44:0/1157952314 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb48003e850 con 0x7fb490153050 2024-04-22T06:31:37.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.010+0000 7fb498746640 1 -- 172.21.15.44:0/1157952314 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4900ce510 con 0x7fb490153050 2024-04-22T06:31:37.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.010+0000 7fb486ffd640 1 -- 172.21.15.44:0/1157952314 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb480043070 con 0x7fb490153050 2024-04-22T06:31:37.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.010+0000 7fb498746640 1 -- 172.21.15.44:0/1157952314 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4900cc980 con 0x7fb490153050 2024-04-22T06:31:37.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.010+0000 7f253f59e640 1 Processor -- start 2024-04-22T06:31:37.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.010+0000 7f69155cc640 1 Processor -- start 2024-04-22T06:31:37.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.011+0000 7f253f59e640 1 -- start start 2024-04-22T06:31:37.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.011+0000 7f28fa228640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f28f40c4ca0 0x7f28f41f1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.011+0000 7f28fa228640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f28f40c4ca0 0x7f28f41f1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.44:45584/0 (socket says 172.21.15.44:45584) 2024-04-22T06:31:37.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.011+0000 7fc494cfd640 1 Processor -- start 2024-04-22T06:31:37.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.011+0000 7f28fa228640 1 -- 172.21.15.44:0/1080919177 learned_addr learned my addr 172.21.15.44:0/1080919177 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:37.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.011+0000 7fc494cfd640 1 -- start start 2024-04-22T06:31:37.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.011+0000 7f28fb22a640 1 --2- 172.21.15.44:0/1080919177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28f41f24a0 0x7f28f414f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.011+0000 7f28fb22a640 1 -- 172.21.15.44:0/1080919177 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f28f4157ed0 con 0x7f28f41f24a0 2024-04-22T06:31:37.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.011+0000 7ff7d5b30640 1 -- 172.21.15.44:0/3093930472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7d00cc580 msgr2=0x7ff7d00cc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.011+0000 7ff7d5b30640 1 --2- 172.21.15.44:0/3093930472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7d00cc580 0x7ff7d00cc9d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff7d00cd8b0 tx=0x7ff7c802bec0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.011+0000 7f28fb22a640 1 -- 172.21.15.44:0/1080919177 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f28f406f5c0 con 0x7f28f40c8600 2024-04-22T06:31:37.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.011+0000 7ff7d5b30640 1 -- 172.21.15.44:0/3093930472 shutdown_connections 2024-04-22T06:31:37.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.011+0000 7ff7d5b30640 1 --2- 172.21.15.44:0/3093930472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7d00cc580 0x7ff7d00cc9d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.011+0000 7ff7d5b30640 1 --2- 172.21.15.44:0/3093930472 >> v2:172.21.15.70:3300/0 conn(0x7ff7d0151de0 0x7ff7d00cde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.011+0000 7ff7d5b30640 1 -- 172.21.15.44:0/3093930472 >> 172.21.15.44:0/3093930472 conn(0x7ff7d0074cb0 msgr2=0x7ff7d00750c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:37.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.011+0000 7f69155cc640 1 -- start start 2024-04-22T06:31:37.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.012+0000 7f28faa29640 1 --2- 172.21.15.44:0/1080919177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28f41f24a0 0x7f28f414f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.012+0000 7f28fb22a640 1 -- 172.21.15.44:0/1080919177 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f28f406f250 con 0x7f28f40c4ca0 2024-04-22T06:31:37.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.012+0000 7f28f9a27640 1 --2- 172.21.15.44:0/1080919177 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28f40c8600 0x7f28f414f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.012+0000 7f253f59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2528004800 0x7f25380e6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.012+0000 7f253f59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25380dd210 0x7f25380db910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.012+0000 7f253f59e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25380dbe50 0x7f25380d2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.012+0000 7fecced92640 1 Processor -- start 2024-04-22T06:31:37.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.012+0000 7f253f59e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2538007810 con 0x7f25380dd210 2024-04-22T06:31:37.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.012+0000 7f253f59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2538006c90 con 0x7f2528004800 2024-04-22T06:31:37.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.012+0000 7f253f59e640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f25380074a0 con 0x7f25380dbe50 2024-04-22T06:31:37.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.012+0000 7f69155cc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69100b52c0 0x7f6910161400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.012+0000 7fc494cfd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4900c5de0 0x7fc490150bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.012+0000 7fc494cfd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4900c88a0 0x7fc4901f0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.012+0000 7fc494cfd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc490152000 0x7fc490073610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.012+0000 7f69155cc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69100c70a0 0x7f69101e8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.012+0000 7f69155cc640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f69100cba20 0x7f69101560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.012+0000 7f690f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69100c70a0 0x7f69101e8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.012+0000 7f690f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69100c70a0 0x7f69101e8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:34212/0 (socket says 172.21.15.44:34212) 2024-04-22T06:31:37.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.012+0000 7fc48e59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4900c5de0 0x7fc490150bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.012+0000 7fc48e59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4900c5de0 0x7fc490150bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.44:45594/0 (socket says 172.21.15.44:45594) 2024-04-22T06:31:37.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.012+0000 7fc48dd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4900c88a0 0x7fc4901f0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.012+0000 7fc48e59c640 1 -- 172.21.15.44:0/1621865285 learned_addr learned my addr 172.21.15.44:0/1621865285 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:37.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.012+0000 7fc494cfd640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc4900c9bd0 con 0x7fc4900c88a0 2024-04-22T06:31:37.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.012+0000 7fc494cfd640 1 -- 172.21.15.44:0/1621865285 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc49015c300 con 0x7fc490152000 2024-04-22T06:31:37.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.012+0000 7fc494cfd640 1 -- 172.21.15.44:0/1621865285 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc49015b5c0 con 0x7fc4900c5de0 2024-04-22T06:31:37.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.012+0000 7fc48ed9d640 1 --2- 172.21.15.44:0/1621865285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc490152000 0x7fc490073610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.012+0000 7efdd1093640 1 Processor -- start 2024-04-22T06:31:37.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.012+0000 7f253e59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2528004800 0x7f25380e6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.012+0000 7f253e59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2528004800 0x7f25380e6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:34230/0 (socket says 172.21.15.44:34230) 2024-04-22T06:31:37.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.012+0000 7f253e59c640 1 -- 172.21.15.44:0/4053817247 learned_addr learned my addr 172.21.15.44:0/4053817247 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:37.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.012+0000 7f28f9a27640 1 -- 172.21.15.44:0/1080919177 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f28f40c4ca0 msgr2=0x7f28f41f1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.012+0000 7f28f9a27640 1 --2- 172.21.15.44:0/1080919177 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f28f40c4ca0 0x7f28f41f1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.012+0000 7f28f9a27640 1 -- 172.21.15.44:0/1080919177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28f41f24a0 msgr2=0x7f28f414f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.012+0000 7f28f9a27640 1 --2- 172.21.15.44:0/1080919177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28f41f24a0 0x7f28f414f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.012+0000 7f28f9a27640 1 -- 172.21.15.44:0/1080919177 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f41f2a50 con 0x7f28f40c8600 2024-04-22T06:31:37.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.012+0000 7f253ed9d640 1 --2- 172.21.15.44:0/4053817247 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25380dbe50 0x7f25380d2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.012+0000 7f690f7fe640 1 -- 172.21.15.44:0/967134035 learned_addr learned my addr 172.21.15.44:0/967134035 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:37.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.012+0000 7f69155cc640 1 -- 172.21.15.44:0/967134035 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6910162e70 con 0x7f69100b52c0 2024-04-22T06:31:37.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.012+0000 7f690ffff640 1 --2- 172.21.15.44:0/967134035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69100b52c0 0x7f6910161400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.013+0000 7f6914dcb640 1 --2- 172.21.15.44:0/967134035 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f69100cba20 0x7f69101560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.013+0000 7f28f9a27640 1 --2- 172.21.15.44:0/1080919177 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28f40c8600 0x7f28f414f360 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f28e400c3c0 tx=0x7f28e400c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:37.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.013+0000 7fc48e59c640 1 -- 172.21.15.44:0/1621865285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc490152000 msgr2=0x7fc490073610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.013+0000 7fc48e59c640 1 --2- 172.21.15.44:0/1621865285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc490152000 0x7fc490073610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.013+0000 7fc48e59c640 1 -- 172.21.15.44:0/1621865285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4900c88a0 msgr2=0x7fc4901f0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.013+0000 7fc48e59c640 1 --2- 172.21.15.44:0/1621865285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4900c88a0 0x7fc4901f0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.013+0000 7fc48e59c640 1 -- 172.21.15.44:0/1621865285 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc490072b70 con 0x7fc4900c5de0 2024-04-22T06:31:37.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.013+0000 7f253dd9b640 1 --2- 172.21.15.44:0/4053817247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25380dd210 0x7f25380db910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.013+0000 7f253e59c640 1 -- 172.21.15.44:0/4053817247 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25380dbe50 msgr2=0x7f25380d2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.013+0000 7f253e59c640 1 --2- 172.21.15.44:0/4053817247 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25380dbe50 0x7f25380d2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.013+0000 7f253e59c640 1 -- 172.21.15.44:0/4053817247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25380dd210 msgr2=0x7f25380db910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.013+0000 7f253e59c640 1 --2- 172.21.15.44:0/4053817247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25380dd210 0x7f25380db910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.013+0000 7f253e59c640 1 -- 172.21.15.44:0/4053817247 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25380dc2d0 con 0x7f2528004800 2024-04-22T06:31:37.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.013+0000 7fecced92640 1 -- start start 2024-04-22T06:31:37.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.013+0000 7fb498746640 1 -- 172.21.15.44:0/1157952314 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb478005290 con 0x7fb490153050 2024-04-22T06:31:37.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.013+0000 7ff7d5b30640 1 -- 172.21.15.44:0/3093930472 shutdown_connections 2024-04-22T06:31:37.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.013+0000 7ff7d5b30640 1 -- 172.21.15.44:0/3093930472 wait complete. 2024-04-22T06:31:37.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.013+0000 7fb486ffd640 1 -- 172.21.15.44:0/1157952314 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb480046070 con 0x7fb490153050 2024-04-22T06:31:37.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.013+0000 7f69155cc640 1 -- 172.21.15.44:0/967134035 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6910162b90 con 0x7f69100c70a0 2024-04-22T06:31:37.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.013+0000 7f69155cc640 1 -- 172.21.15.44:0/967134035 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f6910163150 con 0x7f69100cba20 2024-04-22T06:31:37.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.013+0000 7fc48e59c640 1 --2- 172.21.15.44:0/1621865285 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4900c5de0 0x7fc490150bf0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc48000e870 tx=0x7fc48000ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:37.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.013+0000 7f253e59c640 1 --2- 172.21.15.44:0/4053817247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2528004800 0x7f25380e6a40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2534011bd0 tx=0x7f253400f610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:37.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.013+0000 7efdd1093640 1 -- start start 2024-04-22T06:31:37.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.014+0000 7f252effd640 1 -- 172.21.15.44:0/4053817247 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2534009690 con 0x7f2528004800 2024-04-22T06:31:37.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.014+0000 7f252effd640 1 -- 172.21.15.44:0/4053817247 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2534011cc0 con 0x7f2528004800 2024-04-22T06:31:37.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.014+0000 7f252effd640 1 -- 172.21.15.44:0/4053817247 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2534014640 con 0x7f2528004800 2024-04-22T06:31:37.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.014+0000 7f253f59e640 1 -- 172.21.15.44:0/4053817247 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f253804a8b0 con 0x7f2528004800 2024-04-22T06:31:37.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.014+0000 7fb486ffd640 1 --2- 172.21.15.44:0/1157952314 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb47406cd20 0x7fb47406f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.014+0000 7fb486ffd640 1 -- 172.21.15.44:0/1157952314 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb48003e650 con 0x7fb490153050 2024-04-22T06:31:37.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.014+0000 7f253f59e640 1 -- 172.21.15.44:0/4053817247 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f253804ae50 con 0x7f2528004800 2024-04-22T06:31:37.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.014+0000 7fecced92640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecc0045aa0 0x7fecc0178ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.014+0000 7fb4964e2640 1 --2- 172.21.15.44:0/1157952314 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb47406cd20 0x7fb47406f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.014+0000 7f28eaffd640 1 -- 172.21.15.44:0/1080919177 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e40052d0 con 0x7f28f40c8600 2024-04-22T06:31:37.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.014+0000 7f28eaffd640 1 -- 172.21.15.44:0/1080919177 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28e4005470 con 0x7f28f40c8600 2024-04-22T06:31:37.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.014+0000 7f28eaffd640 1 -- 172.21.15.44:0/1080919177 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e4011a50 con 0x7f28f40c8600 2024-04-22T06:31:37.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.014+0000 7f690f7fe640 1 -- 172.21.15.44:0/967134035 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f69100cba20 msgr2=0x7f69101560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.014+0000 7f690f7fe640 1 --2- 172.21.15.44:0/967134035 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f69100cba20 0x7f69101560b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.014+0000 7f690f7fe640 1 -- 172.21.15.44:0/967134035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69100b52c0 msgr2=0x7f6910161400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.014+0000 7efdd1093640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdcc0c60c0 0x7efdcc0ca090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.014+0000 7efdd1093640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efdcc153090 0x7efdcc0bfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.014+0000 7efdd1093640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efdcc153470 0x7efdcc0c0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.014+0000 7fecced92640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecc0046510 0x7fecc00e4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.014+0000 7efdd1093640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efdcc071400 con 0x7efdcc153090 2024-04-22T06:31:37.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.014+0000 7efdd1093640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7efdcc159c30 con 0x7efdcc0c60c0 2024-04-22T06:31:37.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.014+0000 7efdcad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdcc0c60c0 0x7efdcc0ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.014+0000 7efdd1093640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7efdcc0716f0 con 0x7efdcc153470 2024-04-22T06:31:37.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.014+0000 7efdcad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdcc0c60c0 0x7efdcc0ca090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:34242/0 (socket says 172.21.15.44:34242) 2024-04-22T06:31:37.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.014+0000 7f690f7fe640 1 --2- 172.21.15.44:0/967134035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69100b52c0 0x7f6910161400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.014+0000 7f690f7fe640 1 -- 172.21.15.44:0/967134035 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69100c9c30 con 0x7f69100c70a0 2024-04-22T06:31:37.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.014+0000 7efdcad9d640 1 -- 172.21.15.44:0/4139541940 learned_addr learned my addr 172.21.15.44:0/4139541940 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:37.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.015+0000 7feccdd90640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecc0045aa0 0x7fecc0178ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.015+0000 7feccdd90640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecc0045aa0 0x7fecc0178ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:34256/0 (socket says 172.21.15.44:34256) 2024-04-22T06:31:37.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.014+0000 7efdcb59e640 1 --2- 172.21.15.44:0/4139541940 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efdcc153470 0x7efdcc0c0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.014+0000 7ff7d5b30640 1 Processor -- start 2024-04-22T06:31:37.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.014+0000 7ff7d5b30640 1 -- start start 2024-04-22T06:31:37.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.015+0000 7f690ffff640 1 --2- 172.21.15.44:0/967134035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69100b52c0 0x7f6910161400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:31:37.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.015+0000 7fb4964e2640 1 --2- 172.21.15.44:0/1157952314 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb47406cd20 0x7fb47406f1e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb48c0011e0 tx=0x7fb48c0016f0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:37.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.015+0000 7f28fb22a640 1 -- 172.21.15.44:0/1080919177 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28f40728b0 con 0x7f28f40c8600 2024-04-22T06:31:37.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.015+0000 7efdcb59e640 1 -- 172.21.15.44:0/4139541940 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdcc0c60c0 msgr2=0x7efdcc0ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.015+0000 7efdcb59e640 1 --2- 172.21.15.44:0/4139541940 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdcc0c60c0 0x7efdcc0ca090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.015+0000 7efdcb59e640 1 -- 172.21.15.44:0/4139541940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efdcc153090 msgr2=0x7efdcc0bfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.015+0000 7feccdd90640 1 -- 172.21.15.44:0/61995445 learned_addr learned my addr 172.21.15.44:0/61995445 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:37.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.015+0000 7fecced92640 1 --2- 172.21.15.44:0/61995445 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fecc00dd4d0 0x7fecc004cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.015+0000 7f28fb22a640 1 -- 172.21.15.44:0/1080919177 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28f4072e70 con 0x7f28f40c8600 2024-04-22T06:31:37.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.015+0000 7fecced92640 1 -- 172.21.15.44:0/61995445 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fecc00e3d70 con 0x7fecc0046510 2024-04-22T06:31:37.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.015+0000 7f690f7fe640 1 --2- 172.21.15.44:0/967134035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69100c70a0 0x7f69101e8790 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f690002bde0 tx=0x7f69000046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:37.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.015+0000 7fecced92640 1 -- 172.21.15.44:0/61995445 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fecc00e3a90 con 0x7fecc0045aa0 2024-04-22T06:31:37.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.015+0000 7fecced92640 1 -- 172.21.15.44:0/61995445 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fecc0045ff0 con 0x7fecc00dd4d0 2024-04-22T06:31:37.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.015+0000 7efdcb59e640 1 --2- 172.21.15.44:0/4139541940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efdcc153090 0x7efdcc0bfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.015+0000 7efdcb59e640 1 -- 172.21.15.44:0/4139541940 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdcc0be590 con 0x7efdcc153470 2024-04-22T06:31:37.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.015+0000 7efdcad9d640 1 --2- 172.21.15.44:0/4139541940 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdcc0c60c0 0x7efdcc0ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:31:37.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.015+0000 7feccd58f640 1 --2- 172.21.15.44:0/61995445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecc0046510 0x7fecc00e4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.015+0000 7efdcb59e640 1 --2- 172.21.15.44:0/4139541940 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efdcc153470 0x7efdcc0c0330 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efdbc00c3e0 tx=0x7efdbc00c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:37.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.015+0000 7fecce591640 1 --2- 172.21.15.44:0/61995445 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fecc00dd4d0 0x7fecc004cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.015+0000 7feccdd90640 1 -- 172.21.15.44:0/61995445 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fecc00dd4d0 msgr2=0x7fecc004cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.016+0000 7f690cff9640 1 -- 172.21.15.44:0/967134035 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6900037040 con 0x7f69100c70a0 2024-04-22T06:31:37.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.016+0000 7f690cff9640 1 -- 172.21.15.44:0/967134035 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6900042070 con 0x7f69100c70a0 2024-04-22T06:31:37.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.016+0000 7f690cff9640 1 -- 172.21.15.44:0/967134035 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6900032ca0 con 0x7f69100c70a0 2024-04-22T06:31:37.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.016+0000 7feccdd90640 1 --2- 172.21.15.44:0/61995445 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fecc00dd4d0 0x7fecc004cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.016+0000 7feccdd90640 1 -- 172.21.15.44:0/61995445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecc0046510 msgr2=0x7fecc00e4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.016+0000 7feccdd90640 1 --2- 172.21.15.44:0/61995445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecc0046510 0x7fecc00e4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.016+0000 7feccdd90640 1 -- 172.21.15.44:0/61995445 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc00e4b30 con 0x7fecc0045aa0 2024-04-22T06:31:37.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.016+0000 7f69155cc640 1 -- 172.21.15.44:0/967134035 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69100c9f10 con 0x7f69100c70a0 2024-04-22T06:31:37.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.016+0000 7fecce591640 1 --2- 172.21.15.44:0/61995445 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fecc00dd4d0 0x7fecc004cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:31:37.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.016+0000 7f253f59e640 1 -- 172.21.15.44:0/4053817247 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25380dd780 con 0x7f2528004800 2024-04-22T06:31:37.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.016+0000 7feccdd90640 1 --2- 172.21.15.44:0/61995445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecc0045aa0 0x7fecc0178ff0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fecc80692b0 tx=0x7fecc8076120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:37.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.016+0000 7ff7d5b30640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7d00cc580 0x7ff7d00cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.016+0000 7ff7d5b30640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7d0151de0 0x7ff7d00cfef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.016+0000 7ff7d5b30640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff7d015b9d0 0x7ff7d01f3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.016+0000 7ff7d5b30640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d00d0d80 con 0x7ff7d0151de0 2024-04-22T06:31:37.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.016+0000 7ff7d5b30640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d00d0aa0 con 0x7ff7d00cc580 2024-04-22T06:31:37.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.016+0000 7ff7d4b2e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7d00cc580 0x7ff7d00cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.016+0000 7ff7d4b2e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7d00cc580 0x7ff7d00cf9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:34270/0 (socket says 172.21.15.44:34270) 2024-04-22T06:31:37.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.016+0000 7ff7d4b2e640 1 -- 172.21.15.44:0/1312025098 learned_addr learned my addr 172.21.15.44:0/1312025098 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:37.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.016+0000 7ff7d5b30640 1 -- 172.21.15.44:0/1312025098 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ff7d00d1060 con 0x7ff7d015b9d0 2024-04-22T06:31:37.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.016+0000 7f69155cc640 1 -- 172.21.15.44:0/967134035 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69100ca4d0 con 0x7f69100c70a0 2024-04-22T06:31:37.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.016+0000 7fecbe7fc640 1 -- 172.21.15.44:0/61995445 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc8092070 con 0x7fecc0045aa0 2024-04-22T06:31:37.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.016+0000 7ff7cffff640 1 --2- 172.21.15.44:0/1312025098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7d0151de0 0x7ff7d00cfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.016+0000 7ff7d532f640 1 --2- 172.21.15.44:0/1312025098 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff7d015b9d0 0x7ff7d01f3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.016+0000 7ff7d4b2e640 1 -- 172.21.15.44:0/1312025098 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff7d015b9d0 msgr2=0x7ff7d01f3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.017+0000 7f252effd640 1 -- 172.21.15.44:0/4053817247 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2534012040 con 0x7f2528004800 2024-04-22T06:31:37.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.016+0000 7ff7d4b2e640 1 --2- 172.21.15.44:0/1312025098 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff7d015b9d0 0x7ff7d01f3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.017+0000 7ff7d4b2e640 1 -- 172.21.15.44:0/1312025098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7d0151de0 msgr2=0x7ff7d00cfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.017+0000 7ff7d4b2e640 1 --2- 172.21.15.44:0/1312025098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7d0151de0 0x7ff7d00cfef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.017+0000 7ff7d4b2e640 1 -- 172.21.15.44:0/1312025098 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d014a890 con 0x7ff7d00cc580 2024-04-22T06:31:37.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.017+0000 7f28fb22a640 1 -- 172.21.15.44:0/1080919177 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28f40761b0 con 0x7f28f40c8600 2024-04-22T06:31:37.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.017+0000 7fecbe7fc640 1 -- 172.21.15.44:0/61995445 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecc806bd40 con 0x7fecc0045aa0 2024-04-22T06:31:37.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.017+0000 7fecbe7fc640 1 -- 172.21.15.44:0/61995445 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc8069c30 con 0x7fecc0045aa0 2024-04-22T06:31:37.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.017+0000 7ff7cffff640 1 --2- 172.21.15.44:0/1312025098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7d0151de0 0x7ff7d00cfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:31:37.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.017+0000 7ff7d4b2e640 1 --2- 172.21.15.44:0/1312025098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7d00cc580 0x7ff7d00cf9b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000e9d0 tx=0x7ff7c000ee90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:37.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.017+0000 7f252effd640 1 --2- 172.21.15.44:0/4053817247 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f252006c920 0x7f252006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.017+0000 7fecced92640 1 -- 172.21.15.44:0/61995445 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecc00e4e10 con 0x7fecc0045aa0 2024-04-22T06:31:37.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.017+0000 7fecced92640 1 -- 172.21.15.44:0/61995445 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecc0047f10 con 0x7fecc0045aa0 2024-04-22T06:31:37.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.017+0000 7f252effd640 1 -- 172.21.15.44:0/4053817247 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f253408c660 con 0x7f2528004800 2024-04-22T06:31:37.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.017+0000 7f69155cc640 1 -- 172.21.15.44:0/967134035 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69100761b0 con 0x7f69100c70a0 2024-04-22T06:31:37.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.017+0000 7f28eaffd640 1 -- 172.21.15.44:0/1080919177 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f28e4011bf0 con 0x7f28f40c8600 2024-04-22T06:31:37.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.018+0000 7f253dd9b640 1 --2- 172.21.15.44:0/4053817247 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f252006c920 0x7f252006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.018+0000 7f28eaffd640 1 --2- 172.21.15.44:0/1080919177 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f28d806c940 0x7f28d806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.018+0000 7f28eaffd640 1 -- 172.21.15.44:0/1080919177 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f28e408f570 con 0x7f28f40c8600 2024-04-22T06:31:37.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.018+0000 7f28fa228640 1 --2- 172.21.15.44:0/1080919177 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f28d806c940 0x7f28d806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.018+0000 7f253dd9b640 1 --2- 172.21.15.44:0/4053817247 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f252006c920 0x7f252006ede0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f252800c100 tx=0x7f25400ac000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:37.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.018+0000 7f690cff9640 1 -- 172.21.15.44:0/967134035 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f690003a070 con 0x7f69100c70a0 2024-04-22T06:31:37.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.019+0000 7fecced92640 1 -- 172.21.15.44:0/61995445 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecc0005ec0 con 0x7fecc0045aa0 2024-04-22T06:31:37.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.019+0000 7f28fa228640 1 --2- 172.21.15.44:0/1080919177 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f28d806c940 0x7f28d806ee00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f28f0000c00 tx=0x7f28f003a040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:37.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.019+0000 7f690cff9640 1 --2- 172.21.15.44:0/967134035 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f68fc06c9f0 0x7f68fc06eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.019+0000 7efdb37fe640 1 -- 172.21.15.44:0/4139541940 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdbc017070 con 0x7efdcc153470 2024-04-22T06:31:37.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.019+0000 7efdb37fe640 1 -- 172.21.15.44:0/4139541940 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdbc0052f0 con 0x7efdcc153470 2024-04-22T06:31:37.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.019+0000 7f690cff9640 1 -- 172.21.15.44:0/967134035 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f69000acff0 con 0x7f69100c70a0 2024-04-22T06:31:37.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.019+0000 7efdd1093640 1 -- 172.21.15.44:0/4139541940 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdcc0be870 con 0x7efdcc153470 2024-04-22T06:31:37.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.019+0000 7f690ffff640 1 --2- 172.21.15.44:0/967134035 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f68fc06c9f0 0x7f68fc06eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.019+0000 7efdb37fe640 1 -- 172.21.15.44:0/4139541940 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdbc003850 con 0x7efdcc153470 2024-04-22T06:31:37.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.019+0000 7efdd1093640 1 -- 172.21.15.44:0/4139541940 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7efdcc0bee30 con 0x7efdcc153470 2024-04-22T06:31:37.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.019+0000 7fc47effd640 1 -- 172.21.15.44:0/1621865285 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc480017070 con 0x7fc4900c5de0 2024-04-22T06:31:37.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.019+0000 7fc47effd640 1 -- 172.21.15.44:0/1621865285 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc48000c8e0 con 0x7fc4900c5de0 2024-04-22T06:31:37.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.019+0000 7fc47effd640 1 -- 172.21.15.44:0/1621865285 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48000ccd0 con 0x7fc4900c5de0 2024-04-22T06:31:37.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.019+0000 7fb486ffd640 1 -- 172.21.15.44:0/1157952314 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb480030070 con 0x7fb490153050 2024-04-22T06:31:37.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.019+0000 7fc494cfd640 1 -- 172.21.15.44:0/1621865285 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4901f1850 con 0x7fc4900c5de0 2024-04-22T06:31:37.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.019+0000 7fc494cfd640 1 -- 172.21.15.44:0/1621865285 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc490072e10 con 0x7fc4900c5de0 2024-04-22T06:31:37.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.020+0000 7f690ffff640 1 --2- 172.21.15.44:0/967134035 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f68fc06c9f0 0x7f68fc06eeb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f690400afd0 tx=0x7f6904008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:37.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.020+0000 7efdd1093640 1 -- 172.21.15.44:0/4139541940 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdcc068cc0 con 0x7efdcc153470 2024-04-22T06:31:37.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.020+0000 7fc494cfd640 1 -- 172.21.15.44:0/1621865285 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc490068cc0 con 0x7fc4900c5de0 2024-04-22T06:31:37.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.021+0000 7efdb37fe640 1 -- 172.21.15.44:0/4139541940 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efdbc01a070 con 0x7efdcc153470 2024-04-22T06:31:37.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.021+0000 7efdb37fe640 1 --2- 172.21.15.44:0/4139541940 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7efdb806c920 0x7efdb806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.021+0000 7fc47effd640 1 -- 172.21.15.44:0/1621865285 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc48001a070 con 0x7fc4900c5de0 2024-04-22T06:31:37.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.021+0000 7efdb37fe640 1 -- 172.21.15.44:0/4139541940 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7efdbc08b000 con 0x7efdcc153470 2024-04-22T06:31:37.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.021+0000 7efdcad9d640 1 --2- 172.21.15.44:0/4139541940 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7efdb806c920 0x7efdb806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.021+0000 7fc47effd640 1 --2- 172.21.15.44:0/1621865285 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fc47006c8f0 0x7fc47006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.022+0000 7fc47effd640 1 -- 172.21.15.44:0/1621865285 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc48008c0b0 con 0x7fc4900c5de0 2024-04-22T06:31:37.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.022+0000 7fc48dd9b640 1 --2- 172.21.15.44:0/1621865285 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fc47006c8f0 0x7fc47006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.022+0000 7efdcad9d640 1 --2- 172.21.15.44:0/4139541940 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7efdb806c920 0x7efdb806ede0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efdc402bfe0 tx=0x7efdc4030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:37.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.022+0000 7fc48dd9b640 1 --2- 172.21.15.44:0/1621865285 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fc47006c8f0 0x7fc47006edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc484036000 tx=0x7fc484008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:37.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.023+0000 7ff7cd7fa640 1 -- 172.21.15.44:0/1312025098 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c0017070 con 0x7ff7d00cc580 2024-04-22T06:31:37.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.023+0000 7ff7d5b30640 1 -- 172.21.15.44:0/1312025098 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7d014ab70 con 0x7ff7d00cc580 2024-04-22T06:31:37.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.023+0000 7ff7d5b30640 1 -- 172.21.15.44:0/1312025098 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7d0150b00 con 0x7ff7d00cc580 2024-04-22T06:31:37.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.023+0000 7f252effd640 1 -- 172.21.15.44:0/4053817247 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f25380dd780 con 0x7f2528004800 2024-04-22T06:31:37.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.024+0000 7ff7d5b30640 1 -- 172.21.15.44:0/1312025098 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7d00761b0 con 0x7ff7d00cc580 2024-04-22T06:31:37.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.026+0000 7efdb37fe640 1 -- 172.21.15.44:0/4139541940 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efdbc058b30 con 0x7efdcc153470 2024-04-22T06:31:37.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.027+0000 7f28eaffd640 1 -- 172.21.15.44:0/1080919177 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f28e405d740 con 0x7f28f40c8600 2024-04-22T06:31:37.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.030+0000 7fc47effd640 1 -- 172.21.15.44:0/1621865285 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc490068cc0 con 0x7fc4900c5de0 2024-04-22T06:31:37.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.032+0000 7ff7cd7fa640 1 -- 172.21.15.44:0/1312025098 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7c00094b0 con 0x7ff7d00cc580 2024-04-22T06:31:37.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.032+0000 7ff7cd7fa640 1 -- 172.21.15.44:0/1312025098 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c0021060 con 0x7ff7d00cc580 2024-04-22T06:31:37.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.033+0000 7f690cff9640 1 -- 172.21.15.44:0/967134035 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f690007b150 con 0x7f69100c70a0 2024-04-22T06:31:37.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.034+0000 7fecbe7fc640 1 -- 172.21.15.44:0/61995445 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fecc8069dd0 con 0x7fecc0045aa0 2024-04-22T06:31:37.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.035+0000 7fecbe7fc640 1 --2- 172.21.15.44:0/61995445 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fecac06c9f0 0x7fecac06eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.035+0000 7fecbe7fc640 1 -- 172.21.15.44:0/61995445 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fecc81079e0 con 0x7fecc0045aa0 2024-04-22T06:31:37.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.035+0000 7feccd58f640 1 --2- 172.21.15.44:0/61995445 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fecac06c9f0 0x7fecac06eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.035+0000 7feccd58f640 1 --2- 172.21.15.44:0/61995445 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fecac06c9f0 0x7fecac06eeb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fecc016d8a0 tx=0x7fecc4007e30 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:37.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.036+0000 7ff7cd7fa640 1 -- 172.21.15.44:0/1312025098 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff7c001a070 con 0x7ff7d00cc580 2024-04-22T06:31:37.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.037+0000 7fecbe7fc640 1 -- 172.21.15.44:0/61995445 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fecc809b030 con 0x7fecc0045aa0 2024-04-22T06:31:37.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.037+0000 7ff7cd7fa640 1 --2- 172.21.15.44:0/1312025098 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7ff7bc06c920 0x7ff7bc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:37.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.038+0000 7ff7cd7fa640 1 -- 172.21.15.44:0/1312025098 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff7c0057cd0 con 0x7ff7d00cc580 2024-04-22T06:31:37.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.038+0000 7ff7cffff640 1 --2- 172.21.15.44:0/1312025098 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7ff7bc06c920 0x7ff7bc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:37.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.039+0000 7ff7cffff640 1 --2- 172.21.15.44:0/1312025098 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7ff7bc06c920 0x7ff7bc06ede0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4006fd0 tx=0x7ff7c4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:37.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.042+0000 7ff7cd7fa640 1 -- 172.21.15.44:0/1312025098 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff7c005a330 con 0x7ff7d00cc580 2024-04-22T06:31:37.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.457+0000 7fb498746640 1 -- 172.21.15.44:0/1157952314 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fb478005580 con 0x7fb490153050 2024-04-22T06:31:37.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.457+0000 7fb486ffd640 1 -- 172.21.15.44:0/1157952314 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb48007acc0 con 0x7fb490153050 2024-04-22T06:31:37.457 INFO:teuthology.orchestra.run.smithi044.stdout:55834574858 2024-04-22T06:31:37.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.461+0000 7fb484ff9640 1 -- 172.21.15.44:0/1157952314 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb47406cd20 msgr2=0x7fb47406f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.461+0000 7fb484ff9640 1 --2- 172.21.15.44:0/1157952314 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb47406cd20 0x7fb47406f1e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb48c0011e0 tx=0x7fb48c0016f0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.461+0000 7fb484ff9640 1 -- 172.21.15.44:0/1157952314 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb490153050 msgr2=0x7fb4901538d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.461+0000 7fb484ff9640 1 --2- 172.21.15.44:0/1157952314 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb490153050 0x7fb4901538d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb48002be30 tx=0x7fb4800046c0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.461+0000 7fb484ff9640 1 -- 172.21.15.44:0/1157952314 shutdown_connections 2024-04-22T06:31:37.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.461+0000 7fb484ff9640 1 --2- 172.21.15.44:0/1157952314 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb47406cd20 0x7fb47406f1e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.461+0000 7fb484ff9640 1 --2- 172.21.15.44:0/1157952314 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb490153050 0x7fb4901538d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.461+0000 7fb484ff9640 1 --2- 172.21.15.44:0/1157952314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb49015d1b0 0x7fb4901ef4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.461+0000 7fb484ff9640 1 --2- 172.21.15.44:0/1157952314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4900732a0 0x7fb4900cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.461+0000 7fb484ff9640 1 -- 172.21.15.44:0/1157952314 >> 172.21.15.44:0/1157952314 conn(0x7fb490074cb0 msgr2=0x7fb49015a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:37.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.461+0000 7efdd1093640 1 -- 172.21.15.44:0/4139541940 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7efdcc0cb860 con 0x7efdcc153470 2024-04-22T06:31:37.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.462+0000 7fb484ff9640 1 -- 172.21.15.44:0/1157952314 shutdown_connections 2024-04-22T06:31:37.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.462+0000 7fb484ff9640 1 -- 172.21.15.44:0/1157952314 wait complete. 2024-04-22T06:31:37.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.462+0000 7efdb37fe640 1 -- 172.21.15.44:0/4139541940 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efdbc05ce30 con 0x7efdcc153470 2024-04-22T06:31:37.463 INFO:teuthology.orchestra.run.smithi044.stdout:55834574858 2024-04-22T06:31:37.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.463+0000 7f253f59e640 1 -- 172.21.15.44:0/4053817247 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f2538006640 con 0x7f2528004800 2024-04-22T06:31:37.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.463+0000 7f252effd640 1 -- 172.21.15.44:0/4053817247 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2538006640 con 0x7f2528004800 2024-04-22T06:31:37.464 INFO:teuthology.orchestra.run.smithi044.stdout:55834574857 2024-04-22T06:31:37.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.466+0000 7efdb17fa640 1 -- 172.21.15.44:0/4139541940 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7efdb806c920 msgr2=0x7efdb806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.466+0000 7efdb17fa640 1 --2- 172.21.15.44:0/4139541940 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7efdb806c920 0x7efdb806ede0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efdc402bfe0 tx=0x7efdc4030000 comp rx=0 tx=0).stop 2024-04-22T06:31:37.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.466+0000 7efdb17fa640 1 -- 172.21.15.44:0/4139541940 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efdcc153470 msgr2=0x7efdcc0c0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.466+0000 7efdb17fa640 1 --2- 172.21.15.44:0/4139541940 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efdcc153470 0x7efdcc0c0330 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efdbc00c3e0 tx=0x7efdbc00c8a0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.466+0000 7efdb17fa640 1 -- 172.21.15.44:0/4139541940 shutdown_connections 2024-04-22T06:31:37.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.466+0000 7efdb17fa640 1 --2- 172.21.15.44:0/4139541940 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7efdb806c920 0x7efdb806ede0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.466+0000 7efdb17fa640 1 --2- 172.21.15.44:0/4139541940 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efdcc153470 0x7efdcc0c0330 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.466+0000 7efdb17fa640 1 --2- 172.21.15.44:0/4139541940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efdcc153090 0x7efdcc0bfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.466+0000 7efdb17fa640 1 --2- 172.21.15.44:0/4139541940 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdcc0c60c0 0x7efdcc0ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.466+0000 7efdb17fa640 1 -- 172.21.15.44:0/4139541940 >> 172.21.15.44:0/4139541940 conn(0x7efdcc074cb0 msgr2=0x7efdcc0c7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:37.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.467+0000 7efdb17fa640 1 -- 172.21.15.44:0/4139541940 shutdown_connections 2024-04-22T06:31:37.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.467+0000 7efdb17fa640 1 -- 172.21.15.44:0/4139541940 wait complete. 2024-04-22T06:31:37.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.467+0000 7f253f59e640 1 -- 172.21.15.44:0/4053817247 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f252006c920 msgr2=0x7f252006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.467+0000 7f253f59e640 1 --2- 172.21.15.44:0/4053817247 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f252006c920 0x7f252006ede0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f252800c100 tx=0x7f25400ac000 comp rx=0 tx=0).stop 2024-04-22T06:31:37.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.467+0000 7f253f59e640 1 -- 172.21.15.44:0/4053817247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2528004800 msgr2=0x7f25380e6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.467+0000 7f253f59e640 1 --2- 172.21.15.44:0/4053817247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2528004800 0x7f25380e6a40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2534011bd0 tx=0x7f253400f610 comp rx=0 tx=0).stop 2024-04-22T06:31:37.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.467+0000 7f253f59e640 1 -- 172.21.15.44:0/4053817247 shutdown_connections 2024-04-22T06:31:37.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.467+0000 7f253f59e640 1 --2- 172.21.15.44:0/4053817247 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f252006c920 0x7f252006ede0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.467+0000 7f253f59e640 1 --2- 172.21.15.44:0/4053817247 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25380dbe50 0x7f25380d2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.467+0000 7f253f59e640 1 --2- 172.21.15.44:0/4053817247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25380dd210 0x7f25380db910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.467+0000 7f253f59e640 1 --2- 172.21.15.44:0/4053817247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2528004800 0x7f25380e6a40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.467+0000 7f253f59e640 1 -- 172.21.15.44:0/4053817247 >> 172.21.15.44:0/4053817247 conn(0x7f2538051290 msgr2=0x7f25380d8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:37.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.467+0000 7f253f59e640 1 -- 172.21.15.44:0/4053817247 shutdown_connections 2024-04-22T06:31:37.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.467+0000 7f253f59e640 1 -- 172.21.15.44:0/4053817247 wait complete. 2024-04-22T06:31:37.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.469+0000 7f69155cc640 1 -- 172.21.15.44:0/967134035 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f69100727b0 con 0x7f69100c70a0 2024-04-22T06:31:37.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.470+0000 7f690cff9640 1 -- 172.21.15.44:0/967134035 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f690007aa50 con 0x7f69100c70a0 2024-04-22T06:31:37.473 INFO:teuthology.orchestra.run.smithi044.stdout:55834574858 2024-04-22T06:31:37.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.474+0000 7fc494cfd640 1 -- 172.21.15.44:0/1621865285 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fc4900ce600 con 0x7fc4900c5de0 2024-04-22T06:31:37.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.474+0000 7f69155cc640 1 -- 172.21.15.44:0/967134035 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f68fc06c9f0 msgr2=0x7f68fc06eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.474+0000 7f69155cc640 1 --2- 172.21.15.44:0/967134035 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f68fc06c9f0 0x7f68fc06eeb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f690400afd0 tx=0x7f6904008040 comp rx=0 tx=0).stop 2024-04-22T06:31:37.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.474+0000 7f69155cc640 1 -- 172.21.15.44:0/967134035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69100c70a0 msgr2=0x7f69101e8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.474+0000 7f69155cc640 1 --2- 172.21.15.44:0/967134035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69100c70a0 0x7f69101e8790 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f690002bde0 tx=0x7f69000046c0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.474+0000 7f69155cc640 1 -- 172.21.15.44:0/967134035 shutdown_connections 2024-04-22T06:31:37.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.474+0000 7f69155cc640 1 --2- 172.21.15.44:0/967134035 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f68fc06c9f0 0x7f68fc06eeb0 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f690400afd0 tx=0x7f6904008040 comp rx=0 tx=0).stop 2024-04-22T06:31:37.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.474+0000 7f69155cc640 1 --2- 172.21.15.44:0/967134035 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f69100cba20 0x7f69101560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.474+0000 7f69155cc640 1 --2- 172.21.15.44:0/967134035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69100c70a0 0x7f69101e8790 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f690002bde0 tx=0x7f69000046c0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.474+0000 7f69155cc640 1 --2- 172.21.15.44:0/967134035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69100b52c0 0x7f6910161400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.474+0000 7f69155cc640 1 -- 172.21.15.44:0/967134035 >> 172.21.15.44:0/967134035 conn(0x7f6910075720 msgr2=0x7f69100ce730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:37.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.474+0000 7fc47effd640 1 -- 172.21.15.44:0/1621865285 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc4900ce600 con 0x7fc4900c5de0 2024-04-22T06:31:37.475 INFO:teuthology.orchestra.run.smithi044.stdout:55834574858 2024-04-22T06:31:37.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.475+0000 7f69155cc640 1 -- 172.21.15.44:0/967134035 shutdown_connections 2024-04-22T06:31:37.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.475+0000 7f69155cc640 1 -- 172.21.15.44:0/967134035 wait complete. 2024-04-22T06:31:37.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.476+0000 7f28fb22a640 1 -- 172.21.15.44:0/1080919177 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f28f4156b90 con 0x7f28f40c8600 2024-04-22T06:31:37.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.476+0000 7f28eaffd640 1 -- 172.21.15.44:0/1080919177 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f28e405d040 con 0x7f28f40c8600 2024-04-22T06:31:37.476 INFO:teuthology.orchestra.run.smithi044.stdout:55834574859 2024-04-22T06:31:37.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.477+0000 7fecced92640 1 -- 172.21.15.44:0/61995445 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fecc0033e90 con 0x7fecc0045aa0 2024-04-22T06:31:37.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.478+0000 7ff7d5b30640 1 -- 172.21.15.44:0/1312025098 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7ff7d00cb720 con 0x7ff7d00cc580 2024-04-22T06:31:37.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.478+0000 7fecbe7fc640 1 -- 172.21.15.44:0/61995445 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fecc80d5b40 con 0x7fecc0045aa0 2024-04-22T06:31:37.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.478+0000 7fc494cfd640 1 -- 172.21.15.44:0/1621865285 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fc47006c8f0 msgr2=0x7fc47006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.478+0000 7fc494cfd640 1 --2- 172.21.15.44:0/1621865285 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fc47006c8f0 0x7fc47006edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc484036000 tx=0x7fc484008030 comp rx=0 tx=0).stop 2024-04-22T06:31:37.478 INFO:teuthology.orchestra.run.smithi044.stdout:55834574858 2024-04-22T06:31:37.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.478+0000 7fc494cfd640 1 -- 172.21.15.44:0/1621865285 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4900c5de0 msgr2=0x7fc490150bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.478+0000 7fc494cfd640 1 --2- 172.21.15.44:0/1621865285 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4900c5de0 0x7fc490150bf0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc48000e870 tx=0x7fc48000ed30 comp rx=0 tx=0).stop 2024-04-22T06:31:37.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.478+0000 7fc494cfd640 1 -- 172.21.15.44:0/1621865285 shutdown_connections 2024-04-22T06:31:37.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.478+0000 7fc494cfd640 1 --2- 172.21.15.44:0/1621865285 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fc47006c8f0 0x7fc47006edb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.478+0000 7fc494cfd640 1 --2- 172.21.15.44:0/1621865285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc490152000 0x7fc490073610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.478+0000 7fc494cfd640 1 --2- 172.21.15.44:0/1621865285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4900c88a0 0x7fc4901f0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.478+0000 7fc494cfd640 1 --2- 172.21.15.44:0/1621865285 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4900c5de0 0x7fc490150bf0 secure :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc48000e870 tx=0x7fc48000ed30 comp rx=0 tx=0).stop 2024-04-22T06:31:37.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.478+0000 7fc494cfd640 1 -- 172.21.15.44:0/1621865285 >> 172.21.15.44:0/1621865285 conn(0x7fc490068100 msgr2=0x7fc490144320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:37.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.478+0000 7ff7cd7fa640 1 -- 172.21.15.44:0/1312025098 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff7c0059c30 con 0x7ff7d00cc580 2024-04-22T06:31:37.480 INFO:teuthology.orchestra.run.smithi044.stdout:55834574858 2024-04-22T06:31:37.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.480+0000 7f28fb22a640 1 -- 172.21.15.44:0/1080919177 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f28d806c940 msgr2=0x7f28d806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.480+0000 7f28fb22a640 1 --2- 172.21.15.44:0/1080919177 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f28d806c940 0x7f28d806ee00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f28f0000c00 tx=0x7f28f003a040 comp rx=0 tx=0).stop 2024-04-22T06:31:37.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.480+0000 7f28fb22a640 1 -- 172.21.15.44:0/1080919177 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28f40c8600 msgr2=0x7f28f414f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.480+0000 7f28fb22a640 1 --2- 172.21.15.44:0/1080919177 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28f40c8600 0x7f28f414f360 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f28e400c3c0 tx=0x7f28e400c880 comp rx=0 tx=0).stop 2024-04-22T06:31:37.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.480+0000 7f28fb22a640 1 -- 172.21.15.44:0/1080919177 shutdown_connections 2024-04-22T06:31:37.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.480+0000 7f28fb22a640 1 --2- 172.21.15.44:0/1080919177 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f28d806c940 0x7f28d806ee00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.480+0000 7f28fb22a640 1 --2- 172.21.15.44:0/1080919177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28f41f24a0 0x7f28f414f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.480+0000 7f28fb22a640 1 --2- 172.21.15.44:0/1080919177 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28f40c8600 0x7f28f414f360 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.480+0000 7f28fb22a640 1 --2- 172.21.15.44:0/1080919177 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f28f40c4ca0 0x7f28f41f1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.480+0000 7f28fb22a640 1 -- 172.21.15.44:0/1080919177 >> 172.21.15.44:0/1080919177 conn(0x7f28f40cc840 msgr2=0x7f28f40c8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:37.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.481+0000 7f28fb22a640 1 -- 172.21.15.44:0/1080919177 shutdown_connections 2024-04-22T06:31:37.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.481+0000 7fecced92640 1 -- 172.21.15.44:0/61995445 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fecac06c9f0 msgr2=0x7fecac06eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.481+0000 7fecced92640 1 --2- 172.21.15.44:0/61995445 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fecac06c9f0 0x7fecac06eeb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fecc016d8a0 tx=0x7fecc4007e30 comp rx=0 tx=0).stop 2024-04-22T06:31:37.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.481+0000 7fecced92640 1 -- 172.21.15.44:0/61995445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecc0045aa0 msgr2=0x7fecc0178ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.481+0000 7f28fb22a640 1 -- 172.21.15.44:0/1080919177 wait complete. 2024-04-22T06:31:37.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.481+0000 7fecced92640 1 --2- 172.21.15.44:0/61995445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecc0045aa0 0x7fecc0178ff0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fecc80692b0 tx=0x7fecc8076120 comp rx=0 tx=0).stop 2024-04-22T06:31:37.484 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.5 2024-04-22T06:31:37.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.481+0000 7fecced92640 1 -- 172.21.15.44:0/61995445 shutdown_connections 2024-04-22T06:31:37.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.481+0000 7fecced92640 1 --2- 172.21.15.44:0/61995445 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fecac06c9f0 0x7fecac06eeb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.481+0000 7fecced92640 1 --2- 172.21.15.44:0/61995445 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fecc00dd4d0 0x7fecc004cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.481+0000 7fecced92640 1 --2- 172.21.15.44:0/61995445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecc0046510 0x7fecc00e4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.481+0000 7fecced92640 1 --2- 172.21.15.44:0/61995445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecc0045aa0 0x7fecc0178ff0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.481+0000 7fecced92640 1 -- 172.21.15.44:0/61995445 >> 172.21.15.44:0/61995445 conn(0x7fecc00505c0 msgr2=0x7fecc00dc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:37.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.481+0000 7fecced92640 1 -- 172.21.15.44:0/61995445 shutdown_connections 2024-04-22T06:31:37.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.481+0000 7fecced92640 1 -- 172.21.15.44:0/61995445 wait complete. 2024-04-22T06:31:37.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.481+0000 7fc494cfd640 1 -- 172.21.15.44:0/1621865285 shutdown_connections 2024-04-22T06:31:37.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.481+0000 7fc494cfd640 1 -- 172.21.15.44:0/1621865285 wait complete. 2024-04-22T06:31:37.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.482+0000 7ff7d5b30640 1 -- 172.21.15.44:0/1312025098 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7ff7bc06c920 msgr2=0x7ff7bc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.482+0000 7ff7d5b30640 1 --2- 172.21.15.44:0/1312025098 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7ff7bc06c920 0x7ff7bc06ede0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4006fd0 tx=0x7ff7c4008040 comp rx=0 tx=0).stop 2024-04-22T06:31:37.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.482+0000 7ff7d5b30640 1 -- 172.21.15.44:0/1312025098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7d00cc580 msgr2=0x7ff7d00cf9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:37.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.482+0000 7ff7d5b30640 1 --2- 172.21.15.44:0/1312025098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7d00cc580 0x7ff7d00cf9b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000e9d0 tx=0x7ff7c000ee90 comp rx=0 tx=0).stop 2024-04-22T06:31:37.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.482+0000 7ff7d5b30640 1 -- 172.21.15.44:0/1312025098 shutdown_connections 2024-04-22T06:31:37.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.483+0000 7ff7d5b30640 1 --2- 172.21.15.44:0/1312025098 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7ff7bc06c920 0x7ff7bc06ede0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.483+0000 7ff7d5b30640 1 --2- 172.21.15.44:0/1312025098 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff7d015b9d0 0x7ff7d01f3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.483+0000 7ff7d5b30640 1 --2- 172.21.15.44:0/1312025098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7d0151de0 0x7ff7d00cfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.483+0000 7ff7d5b30640 1 --2- 172.21.15.44:0/1312025098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7d00cc580 0x7ff7d00cf9b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:37.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.483+0000 7ff7d5b30640 1 -- 172.21.15.44:0/1312025098 >> 172.21.15.44:0/1312025098 conn(0x7ff7d0074cb0 msgr2=0x7ff7d00ce730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:37.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.483+0000 7ff7d5b30640 1 -- 172.21.15.44:0/1312025098 shutdown_connections 2024-04-22T06:31:37.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:37.483+0000 7ff7d5b30640 1 -- 172.21.15.44:0/1312025098 wait complete. 2024-04-22T06:31:37.490 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.0 2024-04-22T06:31:37.491 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.2 2024-04-22T06:31:37.497 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.3 2024-04-22T06:31:37.499 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574859 for osd.4 2024-04-22T06:31:37.499 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.6 2024-04-22T06:31:37.503 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.1 2024-04-22T06:31:37.504 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.7 2024-04-22T06:31:38.410 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T06:31:38.413 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T06:31:38.485 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-22T06:31:38.490 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-22T06:31:38.493 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-22T06:31:38.498 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-22T06:31:38.500 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-22T06:31:38.500 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-22T06:31:38.504 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-22T06:31:38.504 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-22T06:31:38.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.652+0000 7f084f59e640 1 Processor -- start 2024-04-22T06:31:38.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.652+0000 7f084f59e640 1 -- start start 2024-04-22T06:31:38.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.653+0000 7f084f59e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08480dd4b0 0x7f08480e7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.653+0000 7f084f59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0848046420 0x7f08480468b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.653+0000 7f084f59e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08480e7670 con 0x7f0848047210 2024-04-22T06:31:38.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.653+0000 7f084f59e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f0848047b20 con 0x7f0848046420 2024-04-22T06:31:38.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.653+0000 7f084f59e640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f08480483e0 con 0x7f08480dd4b0 2024-04-22T06:31:38.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.654+0000 7f084ed9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0848046420 0x7f08480468b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.654+0000 7f084ed9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0848046420 0x7f08480468b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:34272/0 (socket says 172.21.15.44:34272) 2024-04-22T06:31:38.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.654+0000 7f084ed9d640 1 -- 172.21.15.44:0/211251050 learned_addr learned my addr 172.21.15.44:0/211251050 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:38.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.654+0000 7f084e59c640 1 --2- 172.21.15.44:0/211251050 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08480dd4b0 0x7f08480e7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.654+0000 7f084ed9d640 1 -- 172.21.15.44:0/211251050 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08480dd4b0 msgr2=0x7f08480e7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:38.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.654+0000 7f084ed9d640 1 --2- 172.21.15.44:0/211251050 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08480dd4b0 0x7f08480e7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:38.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.654+0000 7f084ed9d640 1 -- 172.21.15.44:0/211251050 >> v1:172.21.15.70:6789/0 conn(0x7f0848047210 legacy=0x7f0848007bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:38.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.654+0000 7f084ed9d640 1 -- 172.21.15.44:0/211251050 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08480dd890 con 0x7f0848046420 2024-04-22T06:31:38.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.656+0000 7f084ed9d640 1 --2- 172.21.15.44:0/211251050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0848046420 0x7f08480468b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f085007c660 tx=0x7f08500a7f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d996e2471b8e0d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:38.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.657+0000 7f084d59a640 1 -- 172.21.15.44:0/211251050 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08500b5070 con 0x7f0848046420 2024-04-22T06:31:38.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.657+0000 7f084d59a640 1 -- 172.21.15.44:0/211251050 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08500a8df0 con 0x7f0848046420 2024-04-22T06:31:38.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.658+0000 7f084f59e640 1 -- 172.21.15.44:0/211251050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0848046420 msgr2=0x7f08480468b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:38.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.658+0000 7f084f59e640 1 --2- 172.21.15.44:0/211251050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0848046420 0x7f08480468b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f085007c660 tx=0x7f08500a7f80 comp rx=0 tx=0).stop 2024-04-22T06:31:38.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.658+0000 7f084f59e640 1 -- 172.21.15.44:0/211251050 shutdown_connections 2024-04-22T06:31:38.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.658+0000 7f084f59e640 1 --2- 172.21.15.44:0/211251050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0848046420 0x7f08480468b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:38.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.658+0000 7f084f59e640 1 --2- 172.21.15.44:0/211251050 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08480dd4b0 0x7f08480e7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:38.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.658+0000 7f084f59e640 1 -- 172.21.15.44:0/211251050 >> 172.21.15.44:0/211251050 conn(0x7f0848051280 msgr2=0x7f0848051690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:38.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.658+0000 7f084f59e640 1 -- 172.21.15.44:0/211251050 shutdown_connections 2024-04-22T06:31:38.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.658+0000 7f084f59e640 1 -- 172.21.15.44:0/211251050 wait complete. 2024-04-22T06:31:38.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.660+0000 7f084f59e640 1 Processor -- start 2024-04-22T06:31:38.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.660+0000 7f084f59e640 1 -- start start 2024-04-22T06:31:38.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.662+0000 7f084f59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0848047210 0x7f08480c6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.662+0000 7f084f59e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08480dd4b0 0x7f0848034650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.662+0000 7f084e59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0848047210 0x7f08480c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.662+0000 7f084e59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0848047210 0x7f08480c6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:34288/0 (socket says 172.21.15.44:34288) 2024-04-22T06:31:38.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.662+0000 7f084e59c640 1 -- 172.21.15.44:0/4174029079 learned_addr learned my addr 172.21.15.44:0/4174029079 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:38.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.662+0000 7f084f59e640 1 --2- 172.21.15.44:0/4174029079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08480c7c00 0x7f08480e5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.662+0000 7f084dd9b640 1 --2- 172.21.15.44:0/4174029079 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08480dd4b0 0x7f0848034650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.662+0000 7f084f59e640 1 -- 172.21.15.44:0/4174029079 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f08480070e0 con 0x7f08480c7c00 2024-04-22T06:31:38.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.662+0000 7f084f59e640 1 -- 172.21.15.44:0/4174029079 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f0848006e00 con 0x7f0848047210 2024-04-22T06:31:38.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.662+0000 7f084f59e640 1 -- 172.21.15.44:0/4174029079 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f08480073c0 con 0x7f08480dd4b0 2024-04-22T06:31:38.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.662+0000 7f084ed9d640 1 --2- 172.21.15.44:0/4174029079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08480c7c00 0x7f08480e5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.663+0000 7f084dd9b640 1 -- 172.21.15.44:0/4174029079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0848047210 msgr2=0x7f08480c6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:38.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.663+0000 7f084dd9b640 1 --2- 172.21.15.44:0/4174029079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0848047210 0x7f08480c6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:38.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.663+0000 7f084dd9b640 1 -- 172.21.15.44:0/4174029079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08480c7c00 msgr2=0x7f08480e5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:38.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.663+0000 7f084dd9b640 1 --2- 172.21.15.44:0/4174029079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08480c7c00 0x7f08480e5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:38.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.663+0000 7f084dd9b640 1 -- 172.21.15.44:0/4174029079 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08480c8100 con 0x7f08480dd4b0 2024-04-22T06:31:38.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.663+0000 7f084ed9d640 1 --2- 172.21.15.44:0/4174029079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08480c7c00 0x7f08480e5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:31:38.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.665+0000 7f084dd9b640 1 --2- 172.21.15.44:0/4174029079 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08480dd4b0 0x7f0848034650 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f084000c3c0 tx=0x7f084000c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:38.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.665+0000 7f083effd640 1 -- 172.21.15.44:0/4174029079 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084005b070 con 0x7f08480dd4b0 2024-04-22T06:31:38.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.665+0000 7f083effd640 1 -- 172.21.15.44:0/4174029079 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08400050d0 con 0x7f08480dd4b0 2024-04-22T06:31:38.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.665+0000 7f083effd640 1 -- 172.21.15.44:0/4174029079 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08400054a0 con 0x7f08480dd4b0 2024-04-22T06:31:38.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.665+0000 7f084f59e640 1 -- 172.21.15.44:0/4174029079 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f084804c250 con 0x7f08480dd4b0 2024-04-22T06:31:38.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.665+0000 7f084f59e640 1 -- 172.21.15.44:0/4174029079 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f08480496b0 con 0x7f08480dd4b0 2024-04-22T06:31:38.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.669+0000 7f083effd640 1 -- 172.21.15.44:0/4174029079 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f084005e070 con 0x7f08480dd4b0 2024-04-22T06:31:38.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.670+0000 7f083effd640 1 --2- 172.21.15.44:0/4174029079 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f083006cb90 0x7f083006f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.670+0000 7f083effd640 1 -- 172.21.15.44:0/4174029079 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f08400cefe0 con 0x7f08480dd4b0 2024-04-22T06:31:38.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.671+0000 7f084f59e640 1 -- 172.21.15.44:0/4174029079 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0834005290 con 0x7f08480dd4b0 2024-04-22T06:31:38.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.671+0000 7f084e59c640 1 --2- 172.21.15.44:0/4174029079 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f083006cb90 0x7f083006f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.672+0000 7f084e59c640 1 --2- 172.21.15.44:0/4174029079 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f083006cb90 0x7f083006f050 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0844006fd0 tx=0x7f0844008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:38.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.680+0000 7f083effd640 1 -- 172.21.15.44:0/4174029079 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f084009c9d0 con 0x7f08480dd4b0 2024-04-22T06:31:38.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.703+0000 7efe3bfff640 1 Processor -- start 2024-04-22T06:31:38.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.703+0000 7efe3bfff640 1 -- start start 2024-04-22T06:31:38.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.704+0000 7efe3bfff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efe34046160 0x7efe340e3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.704+0000 7efe3bfff640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe34046630 con 0x7efe340d9be0 2024-04-22T06:31:38.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.704+0000 7efe3bfff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe34006ed0 con 0x7efe3404bc00 2024-04-22T06:31:38.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.704+0000 7efe3bfff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7efe340d9fc0 con 0x7efe34046160 2024-04-22T06:31:38.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.705+0000 7efe3b7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7efe3404bc00 0x7efe340edac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.44:46236/0 (socket says 172.21.15.44:46236) 2024-04-22T06:31:38.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.705+0000 7efe3b7fe640 1 -- 172.21.15.44:0/3525724267 learned_addr learned my addr 172.21.15.44:0/3525724267 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:38.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.705+0000 7efe3a7fc640 1 --2- 172.21.15.44:0/3525724267 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efe34046160 0x7efe340e3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.705+0000 7efe3a7fc640 1 -- 172.21.15.44:0/3525724267 >> v1:172.21.15.70:6789/0 conn(0x7efe3404bc00 legacy=0x7efe340edac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:38.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.705+0000 7efe3a7fc640 1 -- 172.21.15.44:0/3525724267 >> v1:172.21.15.44:6789/0 conn(0x7efe340d9be0 legacy=0x7efe34046e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:38.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.705+0000 7efe3a7fc640 1 -- 172.21.15.44:0/3525724267 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe340071b0 con 0x7efe34046160 2024-04-22T06:31:38.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.707+0000 7efe3a7fc640 1 --2- 172.21.15.44:0/3525724267 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efe34046160 0x7efe340e3bc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7efe2c009750 tx=0x7efe2c02b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5945f59e9b16576 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:38.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.707+0000 7efe39ffb640 1 -- 172.21.15.44:0/3525724267 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2c039070 con 0x7efe34046160 2024-04-22T06:31:38.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.707+0000 7efe39ffb640 1 -- 172.21.15.44:0/3525724267 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe2c0316f0 con 0x7efe34046160 2024-04-22T06:31:38.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.708+0000 7efe39ffb640 1 -- 172.21.15.44:0/3525724267 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2c031ac0 con 0x7efe34046160 2024-04-22T06:31:38.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.708+0000 7efe3bfff640 1 -- 172.21.15.44:0/3525724267 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efe34046160 msgr2=0x7efe340e3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:38.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.708+0000 7efe3bfff640 1 --2- 172.21.15.44:0/3525724267 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efe34046160 0x7efe340e3bc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7efe2c009750 tx=0x7efe2c02b790 comp rx=0 tx=0).stop 2024-04-22T06:31:38.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.708+0000 7efe3bfff640 1 -- 172.21.15.44:0/3525724267 shutdown_connections 2024-04-22T06:31:38.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.708+0000 7efe3bfff640 1 --2- 172.21.15.44:0/3525724267 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efe34046160 0x7efe340e3bc0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:38.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.708+0000 7efe3bfff640 1 -- 172.21.15.44:0/3525724267 >> 172.21.15.44:0/3525724267 conn(0x7efe34051280 msgr2=0x7efe34051690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:38.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.709+0000 7efe3bfff640 1 -- 172.21.15.44:0/3525724267 shutdown_connections 2024-04-22T06:31:38.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.709+0000 7efe3bfff640 1 -- 172.21.15.44:0/3525724267 wait complete. 2024-04-22T06:31:38.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.710+0000 7efe3bfff640 1 Processor -- start 2024-04-22T06:31:38.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.710+0000 7f2158e25640 1 Processor -- start 2024-04-22T06:31:38.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.710+0000 7f2158e25640 1 -- start start 2024-04-22T06:31:38.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.711+0000 7f3f74e11640 1 Processor -- start 2024-04-22T06:31:38.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.710+0000 7efe3bfff640 1 -- start start 2024-04-22T06:31:38.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.712+0000 7efe3bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe34046160 0x7efe3404abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.712+0000 7f2158e25640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21540c6890 0x7f21540c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.712+0000 7f2158e25640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2154167ff0 con 0x7f21540710b0 2024-04-22T06:31:38.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.712+0000 7f215259c640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f21540710b0 0x7f21541687b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:49510/0 (socket says 172.21.15.44:49510) 2024-04-22T06:31:38.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.712+0000 7f215259c640 1 -- 172.21.15.44:0/1535443614 learned_addr learned my addr 172.21.15.44:0/1535443614 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:38.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.712+0000 7f2152d9d640 1 --2- 172.21.15.44:0/1535443614 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21540c6890 0x7f21540c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.712+0000 7f2158e25640 1 -- 172.21.15.44:0/1535443614 --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21540c7e90 con 0x7f21540c3460 2024-04-22T06:31:38.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.712+0000 7f2158e25640 1 -- 172.21.15.44:0/1535443614 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2154168ec0 con 0x7f21540c6890 2024-04-22T06:31:38.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.712+0000 7efe3bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe3404bc00 0x7efe340df0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.712+0000 7efe3bfff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efe340d9be0 0x7efe340df620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.712+0000 7efe3bfff640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efe34007800 con 0x7efe34046160 2024-04-22T06:31:38.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.712+0000 7efe3bfff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7efe34006c80 con 0x7efe3404bc00 2024-04-22T06:31:38.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.712+0000 7efe3bfff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7efe34047560 con 0x7efe340d9be0 2024-04-22T06:31:38.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.711+0000 7f3f74e11640 1 -- start start 2024-04-22T06:31:38.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.712+0000 7f3f74e11640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3f70151770 0x7f3f70157410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.712+0000 7f3f74e11640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3f700c2d10 0x7f3f7006dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.712+0000 7f3f74e11640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f700c3630 con 0x7f3f700c3120 2024-04-22T06:31:38.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.712+0000 7f3f74e11640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f3f700c38d0 con 0x7f3f70151770 2024-04-22T06:31:38.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.712+0000 7f3f74e11640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f3f70151b50 con 0x7f3f700c2d10 2024-04-22T06:31:38.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.713+0000 7f3f6e59c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3f70151770 0x7f3f70157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.713+0000 7f3f6dd9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3f700c2d10 0x7f3f7006dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.713+0000 7f3f6e59c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3f70151770 0x7f3f70157410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53786/0 (socket says 172.21.15.44:53786) 2024-04-22T06:31:38.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.713+0000 7f3f6e59c640 1 -- 172.21.15.44:0/2620592193 learned_addr learned my addr 172.21.15.44:0/2620592193 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:38.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.713+0000 7efe3a7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe3404bc00 0x7efe340df0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.713+0000 7f215159a640 1 -- 172.21.15.44:0/1535443614 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1205971935 0 0) 0x7f21540c7e90 con 0x7f21540c3460 2024-04-22T06:31:38.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.713+0000 7f215159a640 1 -- 172.21.15.44:0/1535443614 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f213c003520 con 0x7f21540c3460 2024-04-22T06:31:38.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.713+0000 7f215159a640 1 -- 172.21.15.44:0/1535443614 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2235850820 0 0) 0x7f2154167ff0 con 0x7f21540710b0 2024-04-22T06:31:38.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.713+0000 7f215159a640 1 -- 172.21.15.44:0/1535443614 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f21540c7e90 con 0x7f21540710b0 2024-04-22T06:31:38.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.713+0000 7efe3affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe34046160 0x7efe3404abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.713+0000 7efe3affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe34046160 0x7efe3404abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53790/0 (socket says 172.21.15.44:53790) 2024-04-22T06:31:38.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.713+0000 7efe3affd640 1 -- 172.21.15.44:0/3586552148 learned_addr learned my addr 172.21.15.44:0/3586552148 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:38.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.713+0000 7efe3b7fe640 1 --2- 172.21.15.44:0/3586552148 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efe340d9be0 0x7efe340df620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.713+0000 7f212b154640 1 Processor -- start 2024-04-22T06:31:38.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.713+0000 7f215159a640 1 -- 172.21.15.44:0/1535443614 <== mon.1 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2908730293 0 0) 0x7f213c003520 con 0x7f21540c3460 2024-04-22T06:31:38.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.713+0000 7f212b154640 1 -- start start 2024-04-22T06:31:38.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.713+0000 7f215159a640 1 -- 172.21.15.44:0/1535443614 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21540c6890 msgr2=0x7f21540c70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:38.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.713+0000 7f215159a640 1 --2- 172.21.15.44:0/1535443614 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21540c6890 0x7f21540c70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:38.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.714+0000 7f215159a640 1 -- 172.21.15.44:0/1535443614 >> v1:172.21.15.44:6789/0 conn(0x7f21540710b0 legacy=0x7f21541687b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:38.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.714+0000 7f215159a640 1 -- 172.21.15.44:0/1535443614 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f215416c000 con 0x7f21540c3460 2024-04-22T06:31:38.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.714+0000 7f215159a640 1 -- 172.21.15.44:0/1535443614 <== mon.1 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1237339257 0 0) 0x7f2144001670 con 0x7f21540c3460 2024-04-22T06:31:38.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.714+0000 7f215159a640 1 -- 172.21.15.44:0/1535443614 >> v1:172.21.15.70:6789/0 conn(0x7f21540c3460 legacy=0x7f215406f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:38.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.714+0000 7f215159a640 1 --2- 172.21.15.44:0/1535443614 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f213c004ba0 0x7f213c004fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.714+0000 7f215159a640 1 --2- 172.21.15.44:0/1535443614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f213c0037a0 0x7f213c003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.714+0000 7f3f6e59c640 1 -- 172.21.15.44:0/2620592193 >> v2:172.21.15.70:3300/0 conn(0x7f3f700c2d10 msgr2=0x7f3f7006dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:38.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.714+0000 7f3f6e59c640 1 --2- 172.21.15.44:0/2620592193 >> v2:172.21.15.70:3300/0 conn(0x7f3f700c2d10 0x7f3f7006dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:38.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.714+0000 7f3f6e59c640 1 -- 172.21.15.44:0/2620592193 >> v1:172.21.15.44:6789/0 conn(0x7f3f700c3120 legacy=0x7f3f700c1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:38.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.714+0000 7f3f6e59c640 1 -- 172.21.15.44:0/2620592193 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f700c3b70 con 0x7f3f70151770 2024-04-22T06:31:38.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.714+0000 7f080bc77640 1 Processor -- start 2024-04-22T06:31:38.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.714+0000 7f215159a640 1 --2- 172.21.15.44:0/1535443614 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f213c004160 0x7f213c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.714+0000 7f215159a640 1 -- 172.21.15.44:0/1535443614 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2154168ec0 con 0x7f213c0037a0 2024-04-22T06:31:38.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.714+0000 7f215159a640 1 -- 172.21.15.44:0/1535443614 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f213c00bfb0 con 0x7f213c004160 2024-04-22T06:31:38.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.714+0000 7f215159a640 1 -- 172.21.15.44:0/1535443614 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f213c00c180 con 0x7f213c004ba0 2024-04-22T06:31:38.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.715+0000 7f2151d9b640 1 --2- 172.21.15.44:0/1535443614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f213c0037a0 0x7f213c003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.715+0000 7f215259c640 1 --2- 172.21.15.44:0/1535443614 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f213c004ba0 0x7f213c004fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.713+0000 7f3dd0cee640 1 Processor -- start 2024-04-22T06:31:38.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.713+0000 7f3dd0cee640 1 -- start start 2024-04-22T06:31:38.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.715+0000 7f212b154640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f211c04ba50 0x7f211c04be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.715+0000 7f212b154640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f211c0e3080 0x7f211c0e34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.715+0000 7f212b154640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f211c046b60 con 0x7f211c0dd3b0 2024-04-22T06:31:38.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.715+0000 7f212b154640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f211c045440 con 0x7f211c04ba50 2024-04-22T06:31:38.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.715+0000 7f212b154640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f211c045fc0 con 0x7f211c0e3080 2024-04-22T06:31:38.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.715+0000 7f212a953640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f211c0e3080 0x7f211c0e34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.715+0000 7f212a953640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f211c0e3080 0x7f211c0e34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:34326/0 (socket says 172.21.15.44:34326) 2024-04-22T06:31:38.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.715+0000 7f212a953640 1 -- 172.21.15.44:0/2223414531 learned_addr learned my addr 172.21.15.44:0/2223414531 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:38.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.715+0000 7efe3affd640 1 -- 172.21.15.44:0/3586552148 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efe340d9be0 msgr2=0x7efe340df620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:38.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.715+0000 7efe3affd640 1 --2- 172.21.15.44:0/3586552148 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efe340d9be0 0x7efe340df620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:38.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.715+0000 7efe3affd640 1 -- 172.21.15.44:0/3586552148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe3404bc00 msgr2=0x7efe340df0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:38.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.715+0000 7efe3affd640 1 --2- 172.21.15.44:0/3586552148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe3404bc00 0x7efe340df0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:38.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.715+0000 7efe3affd640 1 -- 172.21.15.44:0/3586552148 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe340d41f0 con 0x7efe34046160 2024-04-22T06:31:38.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.715+0000 7f3dd0cee640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3dcc150f70 0x7f3dcc15ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.715+0000 7f3dd0cee640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3dcc0c74f0 0x7f3dcc071370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.715+0000 7f3dd0cee640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dcc0c8ae0 con 0x7f3dcc0c7930 2024-04-22T06:31:38.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.716+0000 7f212a953640 1 -- 172.21.15.44:0/2223414531 >> v2:172.21.15.44:3300/0 conn(0x7f211c04ba50 msgr2=0x7f211c04be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:38.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.716+0000 7f212a953640 1 --2- 172.21.15.44:0/2223414531 >> v2:172.21.15.44:3300/0 conn(0x7f211c04ba50 0x7f211c04be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:38.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.716+0000 7f212a953640 1 -- 172.21.15.44:0/2223414531 >> v1:172.21.15.70:6789/0 conn(0x7f211c0dd3b0 legacy=0x7f211c046450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:38.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.716+0000 7f212a953640 1 -- 172.21.15.44:0/2223414531 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f211c0dd790 con 0x7f211c0e3080 2024-04-22T06:31:38.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.715+0000 7f3dd0cee640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f3dcc0c8dc0 con 0x7f3dcc0c74f0 2024-04-22T06:31:38.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.715+0000 7f3dd0cee640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3dcc0c90a0 con 0x7f3dcc150f70 2024-04-22T06:31:38.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.715+0000 7f3dcb7fe640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3dcc150f70 0x7f3dcc15ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.715+0000 7f3dcb7fe640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3dcc150f70 0x7f3dcc15ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.44:45678/0 (socket says 172.21.15.44:45678) 2024-04-22T06:31:38.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.715+0000 7f3dcb7fe640 1 -- 172.21.15.44:0/1406963565 learned_addr learned my addr 172.21.15.44:0/1406963565 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:38.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.715+0000 7f3dcaffd640 1 --2- 172.21.15.44:0/1406963565 >> v2:172.21.15.44:3300/0 conn(0x7f3dcc0c74f0 0x7f3dcc071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.715+0000 7f3dcb7fe640 1 -- 172.21.15.44:0/1406963565 >> v2:172.21.15.44:3300/0 conn(0x7f3dcc0c74f0 msgr2=0x7f3dcc071370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:38.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.715+0000 7f3dcb7fe640 1 --2- 172.21.15.44:0/1406963565 >> v2:172.21.15.44:3300/0 conn(0x7f3dcc0c74f0 0x7f3dcc071370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:38.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.715+0000 7f080bc77640 1 -- start start 2024-04-22T06:31:38.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.716+0000 7f3dcb7fe640 1 -- 172.21.15.44:0/1406963565 >> v1:172.21.15.44:6789/0 conn(0x7f3dcc0c7930 legacy=0x7f3dcc0cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:38.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.716+0000 7f080bc77640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f0804155210 0x7f080415bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.716+0000 7f080bc77640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08040c8020 0x7f08040ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.716+0000 7f080bc77640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08040cbc30 con 0x7f08041555f0 2024-04-22T06:31:38.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.716+0000 7f080bc77640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f080415c0e0 con 0x7f0804155210 2024-04-22T06:31:38.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.716+0000 7f080bc77640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f080406d2d0 con 0x7f08040c8020 2024-04-22T06:31:38.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.716+0000 7f3dcb7fe640 1 -- 172.21.15.44:0/1406963565 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dcc0c9380 con 0x7f3dcc150f70 2024-04-22T06:31:38.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.716+0000 7f3f6e59c640 1 --2- 172.21.15.44:0/2620592193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f70151770 0x7f3f70157410 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3f700ca940 tx=0x7f3f68030420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5efebd15d0a2d77 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:38.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.716+0000 7f2152d9d640 1 --2- 172.21.15.44:0/1535443614 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f213c004160 0x7f213c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.716+0000 7f0809212640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f08041555f0 0x7f080415e690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.44:46254/0 (socket says 172.21.15.44:46254) 2024-04-22T06:31:38.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.716+0000 7f2151d9b640 1 -- 172.21.15.44:0/1535443614 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f213c004ba0 msgr2=0x7f213c004fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:38.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.716+0000 7f2151d9b640 1 --2- 172.21.15.44:0/1535443614 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f213c004ba0 0x7f213c004fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:38.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.716+0000 7f2151d9b640 1 -- 172.21.15.44:0/1535443614 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f213c004160 msgr2=0x7f213c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:38.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.716+0000 7f2151d9b640 1 --2- 172.21.15.44:0/1535443614 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f213c004160 0x7f213c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:38.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.716+0000 7f2151d9b640 1 -- 172.21.15.44:0/1535443614 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f215416c000 con 0x7f213c0037a0 2024-04-22T06:31:38.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.717+0000 7f0809212640 1 -- 172.21.15.44:0/1125970090 learned_addr learned my addr 172.21.15.44:0/1125970090 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:38.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.717+0000 7f0809a13640 1 --2- 172.21.15.44:0/1125970090 >> v2:172.21.15.44:3300/0 conn(0x7f0804155210 0x7f080415bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.717+0000 7f212a953640 1 --2- 172.21.15.44:0/2223414531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f211c0e3080 0x7f211c0e34b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f212407c660 tx=0x7f21240a7ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=273b64d4bbc48e77 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:38.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.717+0000 7f3dcb7fe640 1 --2- 172.21.15.44:0/1406963565 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3dcc150f70 0x7f3dcc15ae00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4010520 tx=0x7f3dc4034890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43ea83fe23566daf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:38.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.716+0000 7efe3affd640 1 --2- 172.21.15.44:0/3586552148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe34046160 0x7efe3404abe0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efe3000c400 tx=0x7efe3000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:38.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.717+0000 7f080a214640 1 --2- 172.21.15.44:0/1125970090 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08040c8020 0x7f08040ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.717+0000 7f2129150640 1 -- 172.21.15.44:0/2223414531 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21240b5070 con 0x7f211c0e3080 2024-04-22T06:31:38.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.717+0000 7f2129150640 1 -- 172.21.15.44:0/2223414531 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21240a8d60 con 0x7f211c0e3080 2024-04-22T06:31:38.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.717+0000 7f0808a11640 1 -- 172.21.15.44:0/1125970090 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 82045017 0 0) 0x7f08040cbc30 con 0x7f08041555f0 2024-04-22T06:31:38.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.717+0000 7f0808a11640 1 -- 172.21.15.44:0/1125970090 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07f0003520 con 0x7f08041555f0 2024-04-22T06:31:38.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.717+0000 7f0809a13640 1 -- 172.21.15.44:0/1125970090 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08040c8020 msgr2=0x7f08040ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:38.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.717+0000 7f0809a13640 1 --2- 172.21.15.44:0/1125970090 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08040c8020 0x7f08040ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:38.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.717+0000 7f0809a13640 1 -- 172.21.15.44:0/1125970090 >> v1:172.21.15.70:6789/0 conn(0x7f08041555f0 legacy=0x7f080415e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:38.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.717+0000 7f0809a13640 1 -- 172.21.15.44:0/1125970090 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f080406c6c0 con 0x7f0804155210 2024-04-22T06:31:38.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.717+0000 7f3f6d59a640 1 -- 172.21.15.44:0/2620592193 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6803c070 con 0x7f3f70151770 2024-04-22T06:31:38.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.717+0000 7f3f6d59a640 1 -- 172.21.15.44:0/2620592193 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f68037600 con 0x7f3f70151770 2024-04-22T06:31:38.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.718+0000 7f2129150640 1 -- 172.21.15.44:0/2223414531 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21240b0640 con 0x7f211c0e3080 2024-04-22T06:31:38.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.718+0000 7f2151d9b640 1 --2- 172.21.15.44:0/1535443614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f213c0037a0 0x7f213c003c20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2144007f00 tx=0x7f213c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=590092d4854e90d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:38.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.718+0000 7f212b154640 1 -- 172.21.15.44:0/2223414531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f211c0e3080 msgr2=0x7f211c0e34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:38.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.718+0000 7f212b154640 1 --2- 172.21.15.44:0/2223414531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f211c0e3080 0x7f211c0e34b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f212407c660 tx=0x7f21240a7ef0 comp rx=0 tx=0).stop 2024-04-22T06:31:38.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.718+0000 7f212b154640 1 -- 172.21.15.44:0/2223414531 shutdown_connections 2024-04-22T06:31:38.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.718+0000 7f212b154640 1 --2- 172.21.15.44:0/2223414531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f211c0e3080 0x7f211c0e34b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:38.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.718+0000 7f212b154640 1 --2- 172.21.15.44:0/2223414531 >> v2:172.21.15.44:3300/0 conn(0x7f211c04ba50 0x7f211c04be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:38.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.718+0000 7f3f74e11640 1 -- 172.21.15.44:0/2620592193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f70151770 msgr2=0x7f3f70157410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:38.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.718+0000 7f3f74e11640 1 --2- 172.21.15.44:0/2620592193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f70151770 0x7f3f70157410 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3f700ca940 tx=0x7f3f68030420 comp rx=0 tx=0).stop 2024-04-22T06:31:38.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.718+0000 7f597359e640 1 Processor -- start 2024-04-22T06:31:38.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.717+0000 7efe23fff640 1 -- 172.21.15.44:0/3586552148 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe30017070 con 0x7efe34046160 2024-04-22T06:31:38.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.717+0000 7efe23fff640 1 -- 172.21.15.44:0/3586552148 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe300052f0 con 0x7efe34046160 2024-04-22T06:31:38.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.717+0000 7efe23fff640 1 -- 172.21.15.44:0/3586552148 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe30003830 con 0x7efe34046160 2024-04-22T06:31:38.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.718+0000 7efe3bfff640 1 -- 172.21.15.44:0/3586552148 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe340e09a0 con 0x7efe34046160 2024-04-22T06:31:38.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.718+0000 7efe3bfff640 1 -- 172.21.15.44:0/3586552148 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe340d6280 con 0x7efe34046160 2024-04-22T06:31:38.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.718+0000 7f212b154640 1 -- 172.21.15.44:0/2223414531 >> 172.21.15.44:0/2223414531 conn(0x7f211c050170 msgr2=0x7f211c050580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:38.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.718+0000 7f215159a640 1 -- 172.21.15.44:0/1535443614 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2144007ad0 con 0x7f213c0037a0 2024-04-22T06:31:38.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.718+0000 7f215159a640 1 -- 172.21.15.44:0/1535443614 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2144033600 con 0x7f213c0037a0 2024-04-22T06:31:38.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.718+0000 7f215159a640 1 -- 172.21.15.44:0/1535443614 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2144033a70 con 0x7f213c0037a0 2024-04-22T06:31:38.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.718+0000 7f212b154640 1 -- 172.21.15.44:0/2223414531 shutdown_connections 2024-04-22T06:31:38.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.718+0000 7f0809a13640 1 --2- 172.21.15.44:0/1125970090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0804155210 0x7f080415bba0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f08000097a0 tx=0x7f080002b740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd500c14af550c73 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:38.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.719+0000 7f212b154640 1 -- 172.21.15.44:0/2223414531 wait complete. 2024-04-22T06:31:38.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.718+0000 7f597359e640 1 -- start start 2024-04-22T06:31:38.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.719+0000 7f2158e25640 1 -- 172.21.15.44:0/1535443614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f213c0037a0 msgr2=0x7f213c003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:38.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.719+0000 7f2158e25640 1 --2- 172.21.15.44:0/1535443614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f213c0037a0 0x7f213c003c20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2144007f00 tx=0x7f213c005830 comp rx=0 tx=0).stop 2024-04-22T06:31:38.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.719+0000 7f2152d9d640 1 -- 172.21.15.44:0/1535443614 reap_dead start 2024-04-22T06:31:38.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.719+0000 7f2158e25640 1 -- 172.21.15.44:0/1535443614 shutdown_connections 2024-04-22T06:31:38.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.719+0000 7f2158e25640 1 -- 172.21.15.44:0/1535443614 >> 172.21.15.44:0/1535443614 conn(0x7f215406cf00 msgr2=0x7f215406d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:38.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.719+0000 7f3f74e11640 1 -- 172.21.15.44:0/2620592193 shutdown_connections 2024-04-22T06:31:38.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.719+0000 7f3f74e11640 1 --2- 172.21.15.44:0/2620592193 >> v2:172.21.15.70:3300/0 conn(0x7f3f700c2d10 0x7f3f7006dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:38.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.719+0000 7f3f74e11640 1 --2- 172.21.15.44:0/2620592193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f70151770 0x7f3f70157410 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:38.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.719+0000 7f3f74e11640 1 -- 172.21.15.44:0/2620592193 >> 172.21.15.44:0/2620592193 conn(0x7f3f700c99a0 msgr2=0x7f3f700c9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:38.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.719+0000 7f3dca7fc640 1 -- 172.21.15.44:0/1406963565 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc400be60 con 0x7f3dcc150f70 2024-04-22T06:31:38.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.720+0000 7f3f74e11640 1 -- 172.21.15.44:0/2620592193 shutdown_connections 2024-04-22T06:31:38.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.720+0000 7f0808a11640 1 -- 172.21.15.44:0/1125970090 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0800037070 con 0x7f0804155210 2024-04-22T06:31:38.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.720+0000 7f0808a11640 1 -- 172.21.15.44:0/1125970090 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0800030510 con 0x7f0804155210 2024-04-22T06:31:38.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.720+0000 7f2158e25640 1 -- 172.21.15.44:0/1535443614 shutdown_connections 2024-04-22T06:31:38.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.720+0000 7f2158e25640 1 -- 172.21.15.44:0/1535443614 wait complete. 2024-04-22T06:31:38.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.720+0000 7f597359e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f596c0e73c0 0x7f596c007bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.720+0000 7f597359e640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f596c006810 con 0x7f596c008110 2024-04-22T06:31:38.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.720+0000 7f597359e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f596c006af0 con 0x7f596c0dd560 2024-04-22T06:31:38.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.720+0000 7f597359e640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f596c0e6690 con 0x7f596c0e73c0 2024-04-22T06:31:38.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.719+0000 7f3dca7fc640 1 -- 172.21.15.44:0/1406963565 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dc4041070 con 0x7f3dcc150f70 2024-04-22T06:31:38.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.719+0000 7f3dca7fc640 1 -- 172.21.15.44:0/1406963565 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc403b980 con 0x7f3dcc150f70 2024-04-22T06:31:38.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.720+0000 7f5972d9d640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f596c008110 0x7f596c0461b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:49532/0 (socket says 172.21.15.44:49532) 2024-04-22T06:31:38.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.720+0000 7f5972d9d640 1 -- 172.21.15.44:0/1939608007 learned_addr learned my addr 172.21.15.44:0/1939608007 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:38.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.721+0000 7f3dd0cee640 1 -- 172.21.15.44:0/1406963565 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3dcc150f70 msgr2=0x7f3dcc15ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:38.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.721+0000 7f3dd0cee640 1 --2- 172.21.15.44:0/1406963565 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3dcc150f70 0x7f3dcc15ae00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4010520 tx=0x7f3dc4034890 comp rx=0 tx=0).stop 2024-04-22T06:31:38.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.720+0000 7f5971d9b640 1 --2- 172.21.15.44:0/1939608007 >> v2:172.21.15.44:3300/0 conn(0x7f596c0e73c0 0x7f596c007bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.721+0000 7f212b154640 1 Processor -- start 2024-04-22T06:31:38.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.721+0000 7f3f74e11640 1 -- 172.21.15.44:0/2620592193 wait complete. 2024-04-22T06:31:38.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.721+0000 7f597159a640 1 -- 172.21.15.44:0/1939608007 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4284159474 0 0) 0x7f596c006810 con 0x7f596c008110 2024-04-22T06:31:38.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.721+0000 7f5971d9b640 1 -- 172.21.15.44:0/1939608007 >> v1:172.21.15.70:6789/0 conn(0x7f596c0dd560 legacy=0x7f596c04b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:38.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.721+0000 7f5971d9b640 1 -- 172.21.15.44:0/1939608007 >> v1:172.21.15.44:6789/0 conn(0x7f596c008110 legacy=0x7f596c0461b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:38.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.721+0000 7f5971d9b640 1 -- 172.21.15.44:0/1939608007 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f596c0e68e0 con 0x7f596c0e73c0 2024-04-22T06:31:38.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.720+0000 7efe21ffb640 1 -- 172.21.15.44:0/3586552148 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe3404d640 con 0x7efe34046160 2024-04-22T06:31:38.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.722+0000 7f2158e25640 1 Processor -- start 2024-04-22T06:31:38.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.721+0000 7f212b154640 1 -- start start 2024-04-22T06:31:38.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.721+0000 7efe23fff640 1 -- 172.21.15.44:0/3586552148 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efe3001a070 con 0x7efe34046160 2024-04-22T06:31:38.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.721+0000 7efe23fff640 1 --2- 172.21.15.44:0/3586552148 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7efe2806c9c0 0x7efe2806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.722+0000 7efe23fff640 1 -- 172.21.15.44:0/3586552148 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7efe3008b1e0 con 0x7efe34046160 2024-04-22T06:31:38.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.721+0000 7f0808a11640 1 -- 172.21.15.44:0/1125970090 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07fc0019f0 con 0x7f0804155210 2024-04-22T06:31:38.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.722+0000 7f080bc77640 1 -- 172.21.15.44:0/1125970090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0804155210 msgr2=0x7f080415bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:38.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.722+0000 7f080bc77640 1 --2- 172.21.15.44:0/1125970090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0804155210 0x7f080415bba0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f08000097a0 tx=0x7f080002b740 comp rx=0 tx=0).stop 2024-04-22T06:31:38.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.722+0000 7f3dd0cee640 1 -- 172.21.15.44:0/1406963565 shutdown_connections 2024-04-22T06:31:38.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.722+0000 7f3dd0cee640 1 --2- 172.21.15.44:0/1406963565 >> v2:172.21.15.44:3300/0 conn(0x7f3dcc0c74f0 0x7f3dcc071370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:38.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.722+0000 7f3dd0cee640 1 --2- 172.21.15.44:0/1406963565 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3dcc150f70 0x7f3dcc15ae00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:38.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.722+0000 7f3dd0cee640 1 -- 172.21.15.44:0/1406963565 >> 172.21.15.44:0/1406963565 conn(0x7f3dcc06d4d0 msgr2=0x7f3dcc074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:38.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.722+0000 7f212b154640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f211c04ba50 0x7f211c0d4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.722+0000 7f2158e25640 1 -- start start 2024-04-22T06:31:38.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.722+0000 7f212b154640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f211c0dd3b0 0x7f211c0d7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.722+0000 7f212b154640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f211c0e3080 0x7f211c0d56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.722+0000 7f212b154640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f211c045970 con 0x7f211c04ba50 2024-04-22T06:31:38.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.722+0000 7f212b154640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f211c045690 con 0x7f211c0e3080 2024-04-22T06:31:38.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.722+0000 7f212b154640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f211c045c50 con 0x7f211c0dd3b0 2024-04-22T06:31:38.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.722+0000 7efe3a7fc640 1 --2- 172.21.15.44:0/3586552148 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7efe2806c9c0 0x7efe2806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.722+0000 7f3dd0cee640 1 -- 172.21.15.44:0/1406963565 shutdown_connections 2024-04-22T06:31:38.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.722+0000 7f212a953640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f211c0e3080 0x7f211c0d56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.722+0000 7f2129951640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f211c0dd3b0 0x7f211c0d7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.722+0000 7f5971d9b640 1 --2- 172.21.15.44:0/1939608007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f596c0e73c0 0x7f596c007bd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5974076f50 tx=0x7f59740a7eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b90508df0f1a9a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:38.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.722+0000 7f2129951640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f211c0dd3b0 0x7f211c0d7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.44:45694/0 (socket says 172.21.15.44:45694) 2024-04-22T06:31:38.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.722+0000 7f2129951640 1 -- 172.21.15.44:0/1054197972 learned_addr learned my addr 172.21.15.44:0/1054197972 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:38.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.723+0000 7f080bc77640 1 -- 172.21.15.44:0/1125970090 shutdown_connections 2024-04-22T06:31:38.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.723+0000 7f080bc77640 1 --2- 172.21.15.44:0/1125970090 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08040c8020 0x7f08040ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:38.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.723+0000 7f080bc77640 1 --2- 172.21.15.44:0/1125970090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0804155210 0x7f080415bba0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:38.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.723+0000 7f080bc77640 1 -- 172.21.15.44:0/1125970090 >> 172.21.15.44:0/1125970090 conn(0x7f0804073020 msgr2=0x7f0804073430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:38.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.722+0000 7f3dd0cee640 1 -- 172.21.15.44:0/1406963565 wait complete. 2024-04-22T06:31:38.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.723+0000 7f212a152640 1 --2- 172.21.15.44:0/1054197972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f211c04ba50 0x7f211c0d4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.723+0000 7f080bc77640 1 -- 172.21.15.44:0/1125970090 shutdown_connections 2024-04-22T06:31:38.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.723+0000 7f2129951640 1 -- 172.21.15.44:0/1054197972 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f211c0e3080 msgr2=0x7f211c0d56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:38.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.723+0000 7f2129951640 1 --2- 172.21.15.44:0/1054197972 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f211c0e3080 0x7f211c0d56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:38.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.723+0000 7f2129951640 1 -- 172.21.15.44:0/1054197972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f211c04ba50 msgr2=0x7f211c0d4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:38.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.723+0000 7f2129951640 1 --2- 172.21.15.44:0/1054197972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f211c04ba50 0x7f211c0d4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:38.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.723+0000 7f2129951640 1 -- 172.21.15.44:0/1054197972 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f211c0da640 con 0x7f211c0dd3b0 2024-04-22T06:31:38.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.723+0000 7f2129951640 1 --2- 172.21.15.44:0/1054197972 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f211c0dd3b0 0x7f211c0d7350 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f211400b440 tx=0x7f211400b900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:38.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.723+0000 7f080bc77640 1 -- 172.21.15.44:0/1125970090 wait complete. 2024-04-22T06:31:38.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.724+0000 7f2158e25640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f213c0037a0 0x7f2154165e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.724+0000 7f2158e25640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21541f4440 0x7f2154162f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.724+0000 7f211affd640 1 -- 172.21.15.44:0/1054197972 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2114015020 con 0x7f211c0dd3b0 2024-04-22T06:31:38.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.724+0000 7f211affd640 1 -- 172.21.15.44:0/1054197972 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2114011890 con 0x7f211c0dd3b0 2024-04-22T06:31:38.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.724+0000 7f2158e25640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21540ca4b0 0x7f21541fd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.724+0000 7f2151d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21541f4440 0x7f2154162f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.724+0000 7f2151d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21541f4440 0x7f2154162f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53870/0 (socket says 172.21.15.44:53870) 2024-04-22T06:31:38.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.724+0000 7f215259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f213c0037a0 0x7f2154165e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.724+0000 7f3dd0cee640 1 Processor -- start 2024-04-22T06:31:38.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.724+0000 7f2151d9b640 1 -- 172.21.15.44:0/2962537398 learned_addr learned my addr 172.21.15.44:0/2962537398 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:38.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.724+0000 7f2158e25640 1 -- 172.21.15.44:0/2962537398 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f21540c8380 con 0x7f21541f4440 2024-04-22T06:31:38.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.724+0000 7f2158e25640 1 -- 172.21.15.44:0/2962537398 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f215416c2e0 con 0x7f21540ca4b0 2024-04-22T06:31:38.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.724+0000 7f2158e25640 1 -- 172.21.15.44:0/2962537398 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2154200e10 con 0x7f213c0037a0 2024-04-22T06:31:38.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.724+0000 7f211affd640 1 -- 172.21.15.44:0/1054197972 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2114011c40 con 0x7f211c0dd3b0 2024-04-22T06:31:38.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.723+0000 7f3f74e11640 1 Processor -- start 2024-04-22T06:31:38.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.723+0000 7f3f74e11640 1 -- start start 2024-04-22T06:31:38.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.724+0000 7f3f74e11640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f700c2d10 0x7f3f70150a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.724+0000 7f3f74e11640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f700c3120 0x7f3f70158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.724+0000 7f3f74e11640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f70151770 0x7f3f70159990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.724+0000 7f3f6e59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f700c2d10 0x7f3f70150a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.724+0000 7f3f74e11640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3f700c3e50 con 0x7f3f70151770 2024-04-22T06:31:38.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.724+0000 7f3f74e11640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7006d850 con 0x7f3f700c2d10 2024-04-22T06:31:38.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.724+0000 7f3f74e11640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3f700b2f70 con 0x7f3f700c3120 2024-04-22T06:31:38.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.724+0000 7f3f6ed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f70151770 0x7f3f70159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.724+0000 7f3f6e59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f700c2d10 0x7f3f70150a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:34370/0 (socket says 172.21.15.44:34370) 2024-04-22T06:31:38.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.724+0000 7f3f6e59c640 1 -- 172.21.15.44:0/1669669561 learned_addr learned my addr 172.21.15.44:0/1669669561 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:38.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.724+0000 7f3f6dd9b640 1 --2- 172.21.15.44:0/1669669561 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f700c3120 0x7f3f70158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.724+0000 7f212b154640 1 -- 172.21.15.44:0/1054197972 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f211c0da920 con 0x7f211c0dd3b0 2024-04-22T06:31:38.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.725+0000 7f212b154640 1 -- 172.21.15.44:0/1054197972 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f211c0daee0 con 0x7f211c0dd3b0 2024-04-22T06:31:38.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.724+0000 7efe3a7fc640 1 --2- 172.21.15.44:0/3586552148 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7efe2806c9c0 0x7efe2806ee80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efe2c036000 tx=0x7efe2c008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:38.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.725+0000 7f080bc77640 1 Processor -- start 2024-04-22T06:31:38.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.724+0000 7f2152d9d640 1 --2- 172.21.15.44:0/2962537398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21540ca4b0 0x7f21541fd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.725+0000 7f3f6e59c640 1 -- 172.21.15.44:0/1669669561 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f700c3120 msgr2=0x7f3f70158b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:38.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.725+0000 7f3f6e59c640 1 --2- 172.21.15.44:0/1669669561 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f700c3120 0x7f3f70158b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:38.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.725+0000 7f3f6e59c640 1 -- 172.21.15.44:0/1669669561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f70151770 msgr2=0x7f3f70159990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:38.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.725+0000 7f3f6e59c640 1 --2- 172.21.15.44:0/1669669561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f70151770 0x7f3f70159990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:38.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.725+0000 7f3f6e59c640 1 -- 172.21.15.44:0/1669669561 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f70158570 con 0x7f3f700c2d10 2024-04-22T06:31:38.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.725+0000 7f080bc77640 1 -- start start 2024-04-22T06:31:38.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.724+0000 7f3dd0cee640 1 -- start start 2024-04-22T06:31:38.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.725+0000 7f3f6e59c640 1 --2- 172.21.15.44:0/1669669561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f700c2d10 0x7f3f70150a00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3f68008db0 tx=0x7f3f68004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:38.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.726+0000 7f3dd0cee640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3dcc0c74f0 0x7f3dcc071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.726+0000 7f3dd0cee640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3dcc0c7930 0x7f3dcc154ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.726+0000 7f3dd0cee640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3dcc150f70 0x7f3dcc0c4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.726+0000 7f3dd0cee640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc0ccb00 con 0x7f3dcc0c7930 2024-04-22T06:31:38.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.726+0000 7f3dd0cee640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc0c9940 con 0x7f3dcc0c74f0 2024-04-22T06:31:38.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.726+0000 7f3dd0cee640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3dcc0ccc80 con 0x7f3dcc150f70 2024-04-22T06:31:38.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.726+0000 7f3dcb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3dcc0c74f0 0x7f3dcc071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.726+0000 7f080bc77640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08040c8020 0x7f080406ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.726+0000 7f3dcb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3dcc0c74f0 0x7f3dcc071aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:34384/0 (socket says 172.21.15.44:34384) 2024-04-22T06:31:38.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.726+0000 7f080bc77640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08041555f0 0x7f08041582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.726+0000 7f212b154640 1 -- 172.21.15.44:0/1054197972 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f211c0052b0 con 0x7f211c0dd3b0 2024-04-22T06:31:38.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.726+0000 7f3dcb7fe640 1 -- 172.21.15.44:0/2430650292 learned_addr learned my addr 172.21.15.44:0/2430650292 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:38.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.726+0000 7f3dcbfff640 1 --2- 172.21.15.44:0/2430650292 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3dcc150f70 0x7f3dcc0c4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.726+0000 7f080bc77640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0804155a90 0x7f0804152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.725+0000 7f3f5effd640 1 -- 172.21.15.44:0/1669669561 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f68047070 con 0x7f3f700c2d10 2024-04-22T06:31:38.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.725+0000 7f3f5effd640 1 -- 172.21.15.44:0/1669669561 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f68042460 con 0x7f3f700c2d10 2024-04-22T06:31:38.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.726+0000 7f3f5effd640 1 -- 172.21.15.44:0/1669669561 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6803c070 con 0x7f3f700c2d10 2024-04-22T06:31:38.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.726+0000 7f3f74e11640 1 -- 172.21.15.44:0/1669669561 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f70159540 con 0x7f3f700c2d10 2024-04-22T06:31:38.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.727+0000 7f3f74e11640 1 -- 172.21.15.44:0/1669669561 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f701f8a40 con 0x7f3f700c2d10 2024-04-22T06:31:38.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.727+0000 7f080bc77640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f080406cf60 con 0x7f08041555f0 2024-04-22T06:31:38.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.727+0000 7f080bc77640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f080406cc80 con 0x7f08040c8020 2024-04-22T06:31:38.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.727+0000 7f080a214640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0804155a90 0x7f0804152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.727+0000 7f3dcb7fe640 1 -- 172.21.15.44:0/2430650292 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3dcc150f70 msgr2=0x7f3dcc0c4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:38.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.727+0000 7f3dcb7fe640 1 --2- 172.21.15.44:0/2430650292 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3dcc150f70 0x7f3dcc0c4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:38.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.727+0000 7f3dcb7fe640 1 -- 172.21.15.44:0/2430650292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3dcc0c7930 msgr2=0x7f3dcc154ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:38.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.727+0000 7f3dcb7fe640 1 --2- 172.21.15.44:0/2430650292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3dcc0c7930 0x7f3dcc154ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:38.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.727+0000 7f3dcb7fe640 1 -- 172.21.15.44:0/2430650292 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dcc1ed460 con 0x7f3dcc0c74f0 2024-04-22T06:31:38.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.726+0000 7f2151d9b640 1 -- 172.21.15.44:0/2962537398 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f213c0037a0 msgr2=0x7f2154165e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:38.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.726+0000 7f2151d9b640 1 --2- 172.21.15.44:0/2962537398 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f213c0037a0 0x7f2154165e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:38.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.726+0000 7f2151d9b640 1 -- 172.21.15.44:0/2962537398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21540ca4b0 msgr2=0x7f21541fd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:38.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.726+0000 7f2151d9b640 1 --2- 172.21.15.44:0/2962537398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21540ca4b0 0x7f21541fd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:38.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.726+0000 7f2151d9b640 1 -- 172.21.15.44:0/2962537398 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21541f4f60 con 0x7f21541f4440 2024-04-22T06:31:38.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.727+0000 7f080a214640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0804155a90 0x7f0804152de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.44:45750/0 (socket says 172.21.15.44:45750) 2024-04-22T06:31:38.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.727+0000 7f080a214640 1 -- 172.21.15.44:0/649271777 learned_addr learned my addr 172.21.15.44:0/649271777 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:38.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.727+0000 7f211affd640 1 -- 172.21.15.44:0/1054197972 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2114018020 con 0x7f211c0dd3b0 2024-04-22T06:31:38.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.727+0000 7f3dcb7fe640 1 --2- 172.21.15.44:0/2430650292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3dcc0c74f0 0x7f3dcc071aa0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3dc403e040 tx=0x7f3dc4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:38.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.727+0000 7f0809212640 1 --2- 172.21.15.44:0/649271777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08041555f0 0x7f08041582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.727+0000 7f2151d9b640 1 --2- 172.21.15.44:0/2962537398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21541f4440 0x7f2154162f10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f214402c9c0 tx=0x7f2144001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:38.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.727+0000 7f0809a13640 1 --2- 172.21.15.44:0/649271777 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08040c8020 0x7f080406ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.727+0000 7f080bc77640 1 -- 172.21.15.44:0/649271777 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f08040cb7f0 con 0x7f0804155a90 2024-04-22T06:31:38.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.728+0000 7f3db3fff640 1 -- 172.21.15.44:0/2430650292 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4008240 con 0x7f3dcc0c74f0 2024-04-22T06:31:38.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.728+0000 7f3db3fff640 1 -- 172.21.15.44:0/2430650292 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dc400ae50 con 0x7f3dcc0c74f0 2024-04-22T06:31:38.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.728+0000 7f3db3fff640 1 -- 172.21.15.44:0/2430650292 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4008640 con 0x7f3dcc0c74f0 2024-04-22T06:31:38.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.728+0000 7f211affd640 1 --2- 172.21.15.44:0/1054197972 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f210806c9f0 0x7f210806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.728+0000 7f211affd640 1 -- 172.21.15.44:0/1054197972 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f211408bf90 con 0x7f211c0dd3b0 2024-04-22T06:31:38.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.728+0000 7f0809212640 1 -- 172.21.15.44:0/649271777 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0804155a90 msgr2=0x7f0804152de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:38.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.728+0000 7f0809212640 1 --2- 172.21.15.44:0/649271777 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0804155a90 0x7f0804152de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:38.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.728+0000 7f0809212640 1 -- 172.21.15.44:0/649271777 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08040c8020 msgr2=0x7f080406ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:38.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.728+0000 7f3dd0cee640 1 -- 172.21.15.44:0/2430650292 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dcc1ecea0 con 0x7f3dcc0c74f0 2024-04-22T06:31:38.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.728+0000 7f3dd0cee640 1 -- 172.21.15.44:0/2430650292 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dcc070fc0 con 0x7f3dcc0c74f0 2024-04-22T06:31:38.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.728+0000 7f0809212640 1 --2- 172.21.15.44:0/649271777 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08040c8020 0x7f080406ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:38.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.728+0000 7f0809212640 1 -- 172.21.15.44:0/649271777 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08041510f0 con 0x7f08041555f0 2024-04-22T06:31:38.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.728+0000 7f3f74e11640 1 -- 172.21.15.44:0/1669669561 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f700b2d90 con 0x7f3f700c2d10 2024-04-22T06:31:38.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.728+0000 7f080a214640 1 --2- 172.21.15.44:0/649271777 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0804155a90 0x7f0804152de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:31:38.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.728+0000 7f3f5effd640 1 -- 172.21.15.44:0/1669669561 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f6804a070 con 0x7f3f700c2d10 2024-04-22T06:31:38.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.728+0000 7f0809212640 1 --2- 172.21.15.44:0/649271777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08041555f0 0x7f08041582c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f07fc00ca50 tx=0x7f07fc00cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:38.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.728+0000 7f212a152640 1 --2- 172.21.15.44:0/1054197972 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f210806c9f0 0x7f210806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.729+0000 7f212a152640 1 --2- 172.21.15.44:0/1054197972 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f210806c9f0 0x7f210806eeb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2120000f60 tx=0x7f2120009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:38.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.729+0000 7f3f5effd640 1 --2- 172.21.15.44:0/1669669561 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f3f5006c9c0 0x7f3f5006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.729+0000 7f3f5effd640 1 -- 172.21.15.44:0/1669669561 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3f680aee00 con 0x7f3f700c2d10 2024-04-22T06:31:38.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.729+0000 7f3dd0cee640 1 -- 172.21.15.44:0/2430650292 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dcc0761b0 con 0x7f3dcc0c74f0 2024-04-22T06:31:38.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.729+0000 7f3f6dd9b640 1 --2- 172.21.15.44:0/1669669561 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f3f5006c9c0 0x7f3f5006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.730+0000 7f3db3fff640 1 -- 172.21.15.44:0/2430650292 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3dc404e020 con 0x7f3dcc0c74f0 2024-04-22T06:31:38.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.730+0000 7f597159a640 1 -- 172.21.15.44:0/1939608007 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59740b5070 con 0x7f596c0e73c0 2024-04-22T06:31:38.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.730+0000 7f3f6dd9b640 1 --2- 172.21.15.44:0/1669669561 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f3f5006c9c0 0x7f3f5006ee80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3f700cce50 tx=0x7f3f60008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:38.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.730+0000 7f07fa7fc640 1 -- 172.21.15.44:0/649271777 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07fc018070 con 0x7f08041555f0 2024-04-22T06:31:38.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.730+0000 7f597159a640 1 -- 172.21.15.44:0/1939608007 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59740a8d20 con 0x7f596c0e73c0 2024-04-22T06:31:38.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.730+0000 7f597159a640 1 -- 172.21.15.44:0/1939608007 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59740b0640 con 0x7f596c0e73c0 2024-04-22T06:31:38.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.730+0000 7f07fa7fc640 1 -- 172.21.15.44:0/649271777 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07fc00a7a0 con 0x7f08041555f0 2024-04-22T06:31:38.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.730+0000 7f3db3fff640 1 --2- 172.21.15.44:0/2430650292 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f3db806c970 0x7f3db806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.730+0000 7f2142ffd640 1 -- 172.21.15.44:0/2962537398 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2144007ad0 con 0x7f21541f4440 2024-04-22T06:31:38.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.730+0000 7f080bc77640 1 -- 172.21.15.44:0/649271777 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08041f55f0 con 0x7f08041555f0 2024-04-22T06:31:38.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.730+0000 7f2142ffd640 1 -- 172.21.15.44:0/2962537398 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2144005320 con 0x7f21541f4440 2024-04-22T06:31:38.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.730+0000 7f2142ffd640 1 -- 172.21.15.44:0/2962537398 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2144003810 con 0x7f21541f4440 2024-04-22T06:31:38.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.730+0000 7f2158e25640 1 -- 172.21.15.44:0/2962537398 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21541f5240 con 0x7f21541f4440 2024-04-22T06:31:38.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.730+0000 7f2158e25640 1 -- 172.21.15.44:0/2962537398 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2154201340 con 0x7f21541f4440 2024-04-22T06:31:38.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.731+0000 7f080bc77640 1 -- 172.21.15.44:0/649271777 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0804155fa0 con 0x7f08041555f0 2024-04-22T06:31:38.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.730+0000 7f3dcaffd640 1 --2- 172.21.15.44:0/2430650292 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f3db806c970 0x7f3db806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.731+0000 7f3db3fff640 1 -- 172.21.15.44:0/2430650292 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3dc40b68f0 con 0x7f3dcc0c74f0 2024-04-22T06:31:38.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.731+0000 7f3dcaffd640 1 --2- 172.21.15.44:0/2430650292 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f3db806c970 0x7f3db806ee30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc0c5bc0 tx=0x7f3dc0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:38.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.730+0000 7efe23fff640 1 -- 172.21.15.44:0/3586552148 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efe3404d640 con 0x7efe34046160 2024-04-22T06:31:38.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.731+0000 7f597359e640 1 -- 172.21.15.44:0/1939608007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f596c0e73c0 msgr2=0x7f596c007bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:38.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.731+0000 7f597359e640 1 --2- 172.21.15.44:0/1939608007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f596c0e73c0 0x7f596c007bd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5974076f50 tx=0x7f59740a7eb0 comp rx=0 tx=0).stop 2024-04-22T06:31:38.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.732+0000 7f2158e25640 1 -- 172.21.15.44:0/2962537398 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2154068cc0 con 0x7f21541f4440 2024-04-22T06:31:38.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.732+0000 7f2142ffd640 1 -- 172.21.15.44:0/2962537398 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f21440039b0 con 0x7f21541f4440 2024-04-22T06:31:38.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.732+0000 7f07fa7fc640 1 -- 172.21.15.44:0/649271777 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07fc00ab90 con 0x7f08041555f0 2024-04-22T06:31:38.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.733+0000 7f080bc77640 1 -- 172.21.15.44:0/649271777 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0804156380 con 0x7f08041555f0 2024-04-22T06:31:38.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.733+0000 7f597359e640 1 -- 172.21.15.44:0/1939608007 shutdown_connections 2024-04-22T06:31:38.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.733+0000 7f597359e640 1 --2- 172.21.15.44:0/1939608007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f596c0e73c0 0x7f596c007bd0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:38.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.733+0000 7f597359e640 1 -- 172.21.15.44:0/1939608007 >> 172.21.15.44:0/1939608007 conn(0x7f596c04d4a0 msgr2=0x7f596c04d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:38.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.733+0000 7f2142ffd640 1 --2- 172.21.15.44:0/2962537398 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f213406c920 0x7f213406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.733+0000 7f2142ffd640 1 -- 172.21.15.44:0/2962537398 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f21440aeee0 con 0x7f21541f4440 2024-04-22T06:31:38.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.733+0000 7f215259c640 1 --2- 172.21.15.44:0/2962537398 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f213406c920 0x7f213406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.733+0000 7f597359e640 1 -- 172.21.15.44:0/1939608007 shutdown_connections 2024-04-22T06:31:38.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.734+0000 7f215259c640 1 --2- 172.21.15.44:0/2962537398 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f213406c920 0x7f213406ede0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2148005df0 tx=0x7f2148010040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:38.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.733+0000 7f597359e640 1 -- 172.21.15.44:0/1939608007 wait complete. 2024-04-22T06:31:38.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.734+0000 7f07fa7fc640 1 -- 172.21.15.44:0/649271777 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f07fc01b070 con 0x7f08041555f0 2024-04-22T06:31:38.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.734+0000 7f3f5effd640 1 -- 172.21.15.44:0/1669669561 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3f700b2d90 con 0x7f3f700c2d10 2024-04-22T06:31:38.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.734+0000 7f07fa7fc640 1 --2- 172.21.15.44:0/649271777 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f07e806c8f0 0x7f07e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.734+0000 7f07fa7fc640 1 -- 172.21.15.44:0/649271777 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f07fc08b7e0 con 0x7f08041555f0 2024-04-22T06:31:38.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.735+0000 7f0809a13640 1 --2- 172.21.15.44:0/649271777 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f07e806c8f0 0x7f07e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.735+0000 7f0809a13640 1 --2- 172.21.15.44:0/649271777 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f07e806c8f0 0x7f07e806edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0800034000 tx=0x7f0800008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:38.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.735+0000 7f597359e640 1 Processor -- start 2024-04-22T06:31:38.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.735+0000 7f597359e640 1 -- start start 2024-04-22T06:31:38.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.736+0000 7f211affd640 1 -- 172.21.15.44:0/1054197972 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f211401f020 con 0x7f211c0dd3b0 2024-04-22T06:31:38.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.736+0000 7f597359e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f596c008110 0x7f596c17c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.736+0000 7f597359e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f596c0dd560 0x7f596c17b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.737+0000 7f597359e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f596c0e73c0 0x7f596c17b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.737+0000 7f597359e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f596c046e70 con 0x7f596c0e73c0 2024-04-22T06:31:38.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.737+0000 7f597359e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f596c0e6ea0 con 0x7f596c008110 2024-04-22T06:31:38.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.737+0000 7f597359e640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f596c0063f0 con 0x7f596c0dd560 2024-04-22T06:31:38.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.737+0000 7f5971d9b640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f596c0dd560 0x7f596c17b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.737+0000 7f5971d9b640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f596c0dd560 0x7f596c17b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.44:45754/0 (socket says 172.21.15.44:45754) 2024-04-22T06:31:38.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.737+0000 7f5971d9b640 1 -- 172.21.15.44:0/1609845483 learned_addr learned my addr 172.21.15.44:0/1609845483 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:38.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.737+0000 7f597259c640 1 --2- 172.21.15.44:0/1609845483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f596c008110 0x7f596c17c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.737+0000 7f5972d9d640 1 --2- 172.21.15.44:0/1609845483 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f596c0e73c0 0x7f596c17b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.737+0000 7f5972d9d640 1 -- 172.21.15.44:0/1609845483 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f596c0dd560 msgr2=0x7f596c17b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:38.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.737+0000 7f5972d9d640 1 --2- 172.21.15.44:0/1609845483 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f596c0dd560 0x7f596c17b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:38.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.737+0000 7f5972d9d640 1 -- 172.21.15.44:0/1609845483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f596c008110 msgr2=0x7f596c17c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:38.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.737+0000 7f5972d9d640 1 --2- 172.21.15.44:0/1609845483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f596c008110 0x7f596c17c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:38.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.737+0000 7f5972d9d640 1 -- 172.21.15.44:0/1609845483 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f596c17d040 con 0x7f596c0e73c0 2024-04-22T06:31:38.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.737+0000 7f5971d9b640 1 --2- 172.21.15.44:0/1609845483 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f596c0dd560 0x7f596c17b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:31:38.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.738+0000 7f5972d9d640 1 --2- 172.21.15.44:0/1609845483 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f596c0e73c0 0x7f596c17b540 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f596c00c6c0 tx=0x7f596400c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:38.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.738+0000 7f3db3fff640 1 -- 172.21.15.44:0/2430650292 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3dc4084470 con 0x7f3dcc0c74f0 2024-04-22T06:31:38.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.740+0000 7f2142ffd640 1 -- 172.21.15.44:0/2962537398 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f214407ca10 con 0x7f21541f4440 2024-04-22T06:31:38.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.750+0000 7f07fa7fc640 1 -- 172.21.15.44:0/649271777 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0804156380 con 0x7f08041555f0 2024-04-22T06:31:38.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.750+0000 7f5962ffd640 1 -- 172.21.15.44:0/1609845483 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59640074e0 con 0x7f596c0e73c0 2024-04-22T06:31:38.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.750+0000 7f5962ffd640 1 -- 172.21.15.44:0/1609845483 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59640047e0 con 0x7f596c0e73c0 2024-04-22T06:31:38.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.750+0000 7f5962ffd640 1 -- 172.21.15.44:0/1609845483 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5964004bd0 con 0x7f596c0e73c0 2024-04-22T06:31:38.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.750+0000 7f597359e640 1 -- 172.21.15.44:0/1609845483 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f596c17a520 con 0x7f596c0e73c0 2024-04-22T06:31:38.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.750+0000 7f597359e640 1 -- 172.21.15.44:0/1609845483 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f596c0db870 con 0x7f596c0e73c0 2024-04-22T06:31:38.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.751+0000 7f5960ff9640 1 -- 172.21.15.44:0/1609845483 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f596c005700 con 0x7f596c0e73c0 2024-04-22T06:31:38.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.763+0000 7f5962ffd640 1 -- 172.21.15.44:0/1609845483 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5964011070 con 0x7f596c0e73c0 2024-04-22T06:31:38.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.763+0000 7f5962ffd640 1 --2- 172.21.15.44:0/1609845483 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f595406c9c0 0x7f595406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:38.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.764+0000 7f5962ffd640 1 -- 172.21.15.44:0/1609845483 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f596408b8b0 con 0x7f596c0e73c0 2024-04-22T06:31:38.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.764+0000 7f5962ffd640 1 -- 172.21.15.44:0/1609845483 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5964059a40 con 0x7f596c0e73c0 2024-04-22T06:31:38.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.765+0000 7f597259c640 1 --2- 172.21.15.44:0/1609845483 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f595406c9c0 0x7f595406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:38.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:38.771+0000 7f597259c640 1 --2- 172.21.15.44:0/1609845483 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f595406c9c0 0x7f595406ee80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5968004ea0 tx=0x7f5968009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:39.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.085+0000 7f084f59e640 1 -- 172.21.15.44:0/4174029079 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f0834005580 con 0x7f08480dd4b0 2024-04-22T06:31:39.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.085+0000 7f083effd640 1 -- 172.21.15.44:0/4174029079 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f08400a0cd0 con 0x7f08480dd4b0 2024-04-22T06:31:39.085 INFO:teuthology.orchestra.run.smithi044.stdout:55834574858 2024-04-22T06:31:39.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.089+0000 7f083cff9640 1 -- 172.21.15.44:0/4174029079 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f083006cb90 msgr2=0x7f083006f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:39.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.089+0000 7f083cff9640 1 --2- 172.21.15.44:0/4174029079 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f083006cb90 0x7f083006f050 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0844006fd0 tx=0x7f0844008040 comp rx=0 tx=0).stop 2024-04-22T06:31:39.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.089+0000 7f083cff9640 1 -- 172.21.15.44:0/4174029079 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08480dd4b0 msgr2=0x7f0848034650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:39.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.089+0000 7f083cff9640 1 --2- 172.21.15.44:0/4174029079 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08480dd4b0 0x7f0848034650 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f084000c3c0 tx=0x7f084000c880 comp rx=0 tx=0).stop 2024-04-22T06:31:39.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.089+0000 7f083cff9640 1 -- 172.21.15.44:0/4174029079 shutdown_connections 2024-04-22T06:31:39.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.089+0000 7f083cff9640 1 --2- 172.21.15.44:0/4174029079 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f083006cb90 0x7f083006f050 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:39.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.089+0000 7f083cff9640 1 --2- 172.21.15.44:0/4174029079 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08480dd4b0 0x7f0848034650 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:39.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.089+0000 7f083cff9640 1 --2- 172.21.15.44:0/4174029079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08480c7c00 0x7f08480e5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:39.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.089+0000 7f083cff9640 1 --2- 172.21.15.44:0/4174029079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0848047210 0x7f08480c6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:39.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.089+0000 7f083cff9640 1 -- 172.21.15.44:0/4174029079 >> 172.21.15.44:0/4174029079 conn(0x7f0848051280 msgr2=0x7f0848007b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:39.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.089+0000 7f083cff9640 1 -- 172.21.15.44:0/4174029079 shutdown_connections 2024-04-22T06:31:39.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.089+0000 7f083cff9640 1 -- 172.21.15.44:0/4174029079 wait complete. 2024-04-22T06:31:39.114 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.5 2024-04-22T06:31:39.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.136+0000 7f3dd0cee640 1 -- 172.21.15.44:0/2430650292 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f3dcc0761b0 con 0x7f3dcc0c74f0 2024-04-22T06:31:39.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.136+0000 7f3db3fff640 1 -- 172.21.15.44:0/2430650292 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3dc4088770 con 0x7f3dcc0c74f0 2024-04-22T06:31:39.136 INFO:teuthology.orchestra.run.smithi044.stdout:55834574859 2024-04-22T06:31:39.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.139+0000 7f3dd0cee640 1 -- 172.21.15.44:0/2430650292 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f3db806c970 msgr2=0x7f3db806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:39.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.139+0000 7f3dd0cee640 1 --2- 172.21.15.44:0/2430650292 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f3db806c970 0x7f3db806ee30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc0c5bc0 tx=0x7f3dc0008040 comp rx=0 tx=0).stop 2024-04-22T06:31:39.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.139+0000 7f3dd0cee640 1 -- 172.21.15.44:0/2430650292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3dcc0c74f0 msgr2=0x7f3dcc071aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:39.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.139+0000 7f3dd0cee640 1 --2- 172.21.15.44:0/2430650292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3dcc0c74f0 0x7f3dcc071aa0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3dc403e040 tx=0x7f3dc4004a40 comp rx=0 tx=0).stop 2024-04-22T06:31:39.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.139+0000 7f3dd0cee640 1 -- 172.21.15.44:0/2430650292 shutdown_connections 2024-04-22T06:31:39.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.139+0000 7f3dd0cee640 1 --2- 172.21.15.44:0/2430650292 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f3db806c970 0x7f3db806ee30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:39.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.139+0000 7f3dd0cee640 1 --2- 172.21.15.44:0/2430650292 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3dcc150f70 0x7f3dcc0c4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:39.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.139+0000 7f3dd0cee640 1 --2- 172.21.15.44:0/2430650292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3dcc0c7930 0x7f3dcc154ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:39.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.139+0000 7f3dd0cee640 1 --2- 172.21.15.44:0/2430650292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3dcc0c74f0 0x7f3dcc071aa0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:39.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.139+0000 7f3dd0cee640 1 -- 172.21.15.44:0/2430650292 >> 172.21.15.44:0/2430650292 conn(0x7f3dcc06d4d0 msgr2=0x7f3dcc0c3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:39.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.140+0000 7f3dd0cee640 1 -- 172.21.15.44:0/2430650292 shutdown_connections 2024-04-22T06:31:39.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.140+0000 7f3dd0cee640 1 -- 172.21.15.44:0/2430650292 wait complete. 2024-04-22T06:31:39.150 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.7 2024-04-22T06:31:39.151 DEBUG:teuthology.parallel:result is None 2024-04-22T06:31:39.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.170+0000 7f212b154640 1 -- 172.21.15.44:0/1054197972 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f211c0dc350 con 0x7f211c0dd3b0 2024-04-22T06:31:39.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.171+0000 7efe21ffb640 1 -- 172.21.15.44:0/3586552148 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7efe34005700 con 0x7efe34046160 2024-04-22T06:31:39.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.171+0000 7f211affd640 1 -- 172.21.15.44:0/1054197972 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f211400d040 con 0x7f211c0dd3b0 2024-04-22T06:31:39.171 INFO:teuthology.orchestra.run.smithi044.stdout:55834574858 2024-04-22T06:31:39.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.171+0000 7efe23fff640 1 -- 172.21.15.44:0/3586552148 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efe30058cf0 con 0x7efe34046160 2024-04-22T06:31:39.172 INFO:teuthology.orchestra.run.smithi044.stdout:55834574859 2024-04-22T06:31:39.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.174+0000 7f212b154640 1 -- 172.21.15.44:0/1054197972 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f210806c9f0 msgr2=0x7f210806eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:39.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.174+0000 7f212b154640 1 --2- 172.21.15.44:0/1054197972 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f210806c9f0 0x7f210806eeb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2120000f60 tx=0x7f2120009040 comp rx=0 tx=0).stop 2024-04-22T06:31:39.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.174+0000 7f212b154640 1 -- 172.21.15.44:0/1054197972 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f211c0dd3b0 msgr2=0x7f211c0d7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:39.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.174+0000 7f212b154640 1 --2- 172.21.15.44:0/1054197972 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f211c0dd3b0 0x7f211c0d7350 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f211400b440 tx=0x7f211400b900 comp rx=0 tx=0).stop 2024-04-22T06:31:39.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.174+0000 7f212b154640 1 -- 172.21.15.44:0/1054197972 shutdown_connections 2024-04-22T06:31:39.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.174+0000 7f212b154640 1 --2- 172.21.15.44:0/1054197972 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f210806c9f0 0x7f210806eeb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:39.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.174+0000 7f212b154640 1 --2- 172.21.15.44:0/1054197972 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f211c0dd3b0 0x7f211c0d7350 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:39.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.174+0000 7f212b154640 1 --2- 172.21.15.44:0/1054197972 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f211c0e3080 0x7f211c0d56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:39.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.174+0000 7f212b154640 1 --2- 172.21.15.44:0/1054197972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f211c04ba50 0x7f211c0d4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:39.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.174+0000 7f212b154640 1 -- 172.21.15.44:0/1054197972 >> 172.21.15.44:0/1054197972 conn(0x7f211c050170 msgr2=0x7f211c04a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:39.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.174+0000 7f212b154640 1 -- 172.21.15.44:0/1054197972 shutdown_connections 2024-04-22T06:31:39.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.174+0000 7f212b154640 1 -- 172.21.15.44:0/1054197972 wait complete. 2024-04-22T06:31:39.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.174+0000 7efe21ffb640 1 -- 172.21.15.44:0/3586552148 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7efe2806c9c0 msgr2=0x7efe2806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:39.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.174+0000 7efe21ffb640 1 --2- 172.21.15.44:0/3586552148 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7efe2806c9c0 0x7efe2806ee80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efe2c036000 tx=0x7efe2c008030 comp rx=0 tx=0).stop 2024-04-22T06:31:39.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.174+0000 7efe21ffb640 1 -- 172.21.15.44:0/3586552148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe34046160 msgr2=0x7efe3404abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:39.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.174+0000 7efe21ffb640 1 --2- 172.21.15.44:0/3586552148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe34046160 0x7efe3404abe0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efe3000c400 tx=0x7efe3000c8c0 comp rx=0 tx=0).stop 2024-04-22T06:31:39.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.174+0000 7efe21ffb640 1 -- 172.21.15.44:0/3586552148 shutdown_connections 2024-04-22T06:31:39.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.174+0000 7efe21ffb640 1 --2- 172.21.15.44:0/3586552148 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7efe2806c9c0 0x7efe2806ee80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:39.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.174+0000 7efe21ffb640 1 --2- 172.21.15.44:0/3586552148 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efe340d9be0 0x7efe340df620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:39.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.174+0000 7efe21ffb640 1 --2- 172.21.15.44:0/3586552148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe3404bc00 0x7efe340df0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:39.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.174+0000 7efe21ffb640 1 --2- 172.21.15.44:0/3586552148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe34046160 0x7efe3404abe0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:39.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.174+0000 7efe21ffb640 1 -- 172.21.15.44:0/3586552148 >> 172.21.15.44:0/3586552148 conn(0x7efe34051280 msgr2=0x7efe34050960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:39.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.175+0000 7efe21ffb640 1 -- 172.21.15.44:0/3586552148 shutdown_connections 2024-04-22T06:31:39.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.175+0000 7efe21ffb640 1 -- 172.21.15.44:0/3586552148 wait complete. 2024-04-22T06:31:39.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.177+0000 7f2158e25640 1 -- 172.21.15.44:0/2962537398 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f21540730e0 con 0x7f21541f4440 2024-04-22T06:31:39.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.178+0000 7f2142ffd640 1 -- 172.21.15.44:0/2962537398 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2144080d10 con 0x7f21541f4440 2024-04-22T06:31:39.177 INFO:teuthology.orchestra.run.smithi044.stdout:55834574859 2024-04-22T06:31:39.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.180+0000 7f2158e25640 1 -- 172.21.15.44:0/2962537398 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f213406c920 msgr2=0x7f213406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:39.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.180+0000 7f2158e25640 1 --2- 172.21.15.44:0/2962537398 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f213406c920 0x7f213406ede0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2148005df0 tx=0x7f2148010040 comp rx=0 tx=0).stop 2024-04-22T06:31:39.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.180+0000 7f2158e25640 1 -- 172.21.15.44:0/2962537398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21541f4440 msgr2=0x7f2154162f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:39.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.180+0000 7f2158e25640 1 --2- 172.21.15.44:0/2962537398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21541f4440 0x7f2154162f10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f214402c9c0 tx=0x7f2144001970 comp rx=0 tx=0).stop 2024-04-22T06:31:39.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.180+0000 7f2158e25640 1 -- 172.21.15.44:0/2962537398 shutdown_connections 2024-04-22T06:31:39.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.180+0000 7f2158e25640 1 --2- 172.21.15.44:0/2962537398 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f213406c920 0x7f213406ede0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:39.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.180+0000 7f2158e25640 1 --2- 172.21.15.44:0/2962537398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21540ca4b0 0x7f21541fd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:39.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.180+0000 7f2158e25640 1 --2- 172.21.15.44:0/2962537398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21541f4440 0x7f2154162f10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:39.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.180+0000 7f2158e25640 1 --2- 172.21.15.44:0/2962537398 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f213c0037a0 0x7f2154165e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:39.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.180+0000 7f2158e25640 1 -- 172.21.15.44:0/2962537398 >> 172.21.15.44:0/2962537398 conn(0x7f215406cf00 msgr2=0x7f21541687b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:39.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.180+0000 7f2158e25640 1 -- 172.21.15.44:0/2962537398 shutdown_connections 2024-04-22T06:31:39.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.180+0000 7f2158e25640 1 -- 172.21.15.44:0/2962537398 wait complete. 2024-04-22T06:31:39.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.181+0000 7f3f74e11640 1 -- 172.21.15.44:0/1669669561 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f3f700761b0 con 0x7f3f700c2d10 2024-04-22T06:31:39.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.182+0000 7f3f5effd640 1 -- 172.21.15.44:0/1669669561 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3f6807c910 con 0x7f3f700c2d10 2024-04-22T06:31:39.182 INFO:teuthology.orchestra.run.smithi044.stdout:55834574859 2024-04-22T06:31:39.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.184+0000 7f080bc77640 1 -- 172.21.15.44:0/649271777 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f0804068140 con 0x7f08041555f0 2024-04-22T06:31:39.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.184+0000 7f07fa7fc640 1 -- 172.21.15.44:0/649271777 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0804068140 con 0x7f08041555f0 2024-04-22T06:31:39.184 INFO:teuthology.orchestra.run.smithi044.stdout:55834574859 2024-04-22T06:31:39.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.185+0000 7f3f74e11640 1 -- 172.21.15.44:0/1669669561 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f3f5006c9c0 msgr2=0x7f3f5006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:39.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.185+0000 7f3f74e11640 1 --2- 172.21.15.44:0/1669669561 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f3f5006c9c0 0x7f3f5006ee80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3f700cce50 tx=0x7f3f60008040 comp rx=0 tx=0).stop 2024-04-22T06:31:39.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.185+0000 7f3f74e11640 1 -- 172.21.15.44:0/1669669561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f700c2d10 msgr2=0x7f3f70150a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:39.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.185+0000 7f3f74e11640 1 --2- 172.21.15.44:0/1669669561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f700c2d10 0x7f3f70150a00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3f68008db0 tx=0x7f3f68004a40 comp rx=0 tx=0).stop 2024-04-22T06:31:39.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.185+0000 7f3f74e11640 1 -- 172.21.15.44:0/1669669561 shutdown_connections 2024-04-22T06:31:39.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.185+0000 7f3f74e11640 1 --2- 172.21.15.44:0/1669669561 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f3f5006c9c0 0x7f3f5006ee80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:39.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.185+0000 7f3f74e11640 1 --2- 172.21.15.44:0/1669669561 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3f700c3120 0x7f3f70158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:39.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.185+0000 7f3f74e11640 1 --2- 172.21.15.44:0/1669669561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f70151770 0x7f3f70159990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:39.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.185+0000 7f3f74e11640 1 --2- 172.21.15.44:0/1669669561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f700c2d10 0x7f3f70150a00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:39.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.185+0000 7f3f74e11640 1 -- 172.21.15.44:0/1669669561 >> 172.21.15.44:0/1669669561 conn(0x7f3f700c99a0 msgr2=0x7f3f7014ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:39.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.185+0000 7f3f74e11640 1 -- 172.21.15.44:0/1669669561 shutdown_connections 2024-04-22T06:31:39.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.186+0000 7f3f74e11640 1 -- 172.21.15.44:0/1669669561 wait complete. 2024-04-22T06:31:39.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.188+0000 7f080bc77640 1 -- 172.21.15.44:0/649271777 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f07e806c8f0 msgr2=0x7f07e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:39.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.188+0000 7f080bc77640 1 --2- 172.21.15.44:0/649271777 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f07e806c8f0 0x7f07e806edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0800034000 tx=0x7f0800008030 comp rx=0 tx=0).stop 2024-04-22T06:31:39.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.188+0000 7f080bc77640 1 -- 172.21.15.44:0/649271777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08041555f0 msgr2=0x7f08041582c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:39.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.188+0000 7f080bc77640 1 --2- 172.21.15.44:0/649271777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08041555f0 0x7f08041582c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f07fc00ca50 tx=0x7f07fc00cf10 comp rx=0 tx=0).stop 2024-04-22T06:31:39.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.188+0000 7f080bc77640 1 -- 172.21.15.44:0/649271777 shutdown_connections 2024-04-22T06:31:39.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.188+0000 7f080bc77640 1 --2- 172.21.15.44:0/649271777 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f07e806c8f0 0x7f07e806edb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:39.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.188+0000 7f080bc77640 1 --2- 172.21.15.44:0/649271777 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0804155a90 0x7f0804152de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:39.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.188+0000 7f080bc77640 1 --2- 172.21.15.44:0/649271777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08041555f0 0x7f08041582c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:39.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.188+0000 7f080bc77640 1 --2- 172.21.15.44:0/649271777 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08040c8020 0x7f080406ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:39.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.188+0000 7f080bc77640 1 -- 172.21.15.44:0/649271777 >> 172.21.15.44:0/649271777 conn(0x7f0804073020 msgr2=0x7f08040cd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:39.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.188+0000 7f080bc77640 1 -- 172.21.15.44:0/649271777 shutdown_connections 2024-04-22T06:31:39.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.188+0000 7f080bc77640 1 -- 172.21.15.44:0/649271777 wait complete. 2024-04-22T06:31:39.191 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.2 2024-04-22T06:31:39.191 DEBUG:teuthology.parallel:result is None 2024-04-22T06:31:39.195 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.6 2024-04-22T06:31:39.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.195+0000 7f5960ff9640 1 -- 172.21.15.44:0/1609845483 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f596c045d80 con 0x7f596c0e73c0 2024-04-22T06:31:39.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.196+0000 7f5962ffd640 1 -- 172.21.15.44:0/1609845483 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5964059340 con 0x7f596c0e73c0 2024-04-22T06:31:39.196 DEBUG:teuthology.parallel:result is None 2024-04-22T06:31:39.197 INFO:teuthology.orchestra.run.smithi044.stdout:55834574860 2024-04-22T06:31:39.198 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.0 2024-04-22T06:31:39.198 DEBUG:teuthology.parallel:result is None 2024-04-22T06:31:39.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.199+0000 7f5960ff9640 1 -- 172.21.15.44:0/1609845483 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f595406c9c0 msgr2=0x7f595406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:39.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.199+0000 7f5960ff9640 1 --2- 172.21.15.44:0/1609845483 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f595406c9c0 0x7f595406ee80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5968004ea0 tx=0x7f5968009040 comp rx=0 tx=0).stop 2024-04-22T06:31:39.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.199+0000 7f5960ff9640 1 -- 172.21.15.44:0/1609845483 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f596c0e73c0 msgr2=0x7f596c17b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:39.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.199+0000 7f5960ff9640 1 --2- 172.21.15.44:0/1609845483 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f596c0e73c0 0x7f596c17b540 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f596c00c6c0 tx=0x7f596400c870 comp rx=0 tx=0).stop 2024-04-22T06:31:39.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.199+0000 7f5960ff9640 1 -- 172.21.15.44:0/1609845483 shutdown_connections 2024-04-22T06:31:39.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.199+0000 7f5960ff9640 1 --2- 172.21.15.44:0/1609845483 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f595406c9c0 0x7f595406ee80 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:39.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.199+0000 7f5960ff9640 1 --2- 172.21.15.44:0/1609845483 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f596c0dd560 0x7f596c17b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:39.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.199+0000 7f5960ff9640 1 --2- 172.21.15.44:0/1609845483 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f596c0e73c0 0x7f596c17b540 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:39.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.199+0000 7f5960ff9640 1 --2- 172.21.15.44:0/1609845483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f596c008110 0x7f596c17c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:39.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.199+0000 7f5960ff9640 1 -- 172.21.15.44:0/1609845483 >> 172.21.15.44:0/1609845483 conn(0x7f596c04d4a0 msgr2=0x7f596c0d8540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:39.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.199+0000 7f5960ff9640 1 -- 172.21.15.44:0/1609845483 shutdown_connections 2024-04-22T06:31:39.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:39.199+0000 7f5960ff9640 1 -- 172.21.15.44:0/1609845483 wait complete. 2024-04-22T06:31:39.203 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.3 2024-04-22T06:31:39.203 DEBUG:teuthology.parallel:result is None 2024-04-22T06:31:39.205 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.1 2024-04-22T06:31:39.205 DEBUG:teuthology.parallel:result is None 2024-04-22T06:31:39.211 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.4 2024-04-22T06:31:39.211 DEBUG:teuthology.parallel:result is None 2024-04-22T06:31:40.116 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-22T06:31:40.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.230+0000 7fba88621640 1 Processor -- start 2024-04-22T06:31:40.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.230+0000 7fba88621640 1 -- start start 2024-04-22T06:31:40.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.231+0000 7fba88621640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba800cb6b0 0x7fba8015e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:40.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.231+0000 7fba88621640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fba800cba90 0x7fba80071880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:40.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.231+0000 7fba88621640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba8015b4b0 con 0x7fba8006c2f0 2024-04-22T06:31:40.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.231+0000 7fba88621640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fba8015b790 con 0x7fba800cba90 2024-04-22T06:31:40.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.231+0000 7fba88621640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fba8015ba70 con 0x7fba800cb6b0 2024-04-22T06:31:40.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.231+0000 7fba86bbe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fba800cba90 0x7fba80071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:40.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.231+0000 7fba863bd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba800cb6b0 0x7fba8015e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:40.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.231+0000 7fba85bbc640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fba8006c2f0 0x7fba80071170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.44:46276/0 (socket says 172.21.15.44:46276) 2024-04-22T06:31:40.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.231+0000 7fba85bbc640 1 -- 172.21.15.44:0/1599674510 learned_addr learned my addr 172.21.15.44:0/1599674510 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:40.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.232+0000 7fba86bbe640 1 -- 172.21.15.44:0/1599674510 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba800cb6b0 msgr2=0x7fba8015e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:40.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.232+0000 7fba86bbe640 1 --2- 172.21.15.44:0/1599674510 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba800cb6b0 0x7fba8015e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:40.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.232+0000 7fba86bbe640 1 -- 172.21.15.44:0/1599674510 >> v1:172.21.15.70:6789/0 conn(0x7fba8006c2f0 legacy=0x7fba80071170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:40.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.232+0000 7fba86bbe640 1 -- 172.21.15.44:0/1599674510 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba8015bd50 con 0x7fba800cba90 2024-04-22T06:31:40.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.232+0000 7fba853bb640 1 -- 172.21.15.44:0/1599674510 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 235020070 0 0) 0x7fba8015b4b0 con 0x7fba8006c2f0 2024-04-22T06:31:40.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.232+0000 7fba863bd640 1 --2- 172.21.15.44:0/1599674510 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba800cb6b0 0x7fba8015e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:31:40.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.232+0000 7fba86bbe640 1 --2- 172.21.15.44:0/1599674510 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba800cba90 0x7fba80071880 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fba700097a0 tx=0x7fba7002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=791e50b1860fec4a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:40.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.233+0000 7fba853bb640 1 -- 172.21.15.44:0/1599674510 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba70035020 con 0x7fba800cba90 2024-04-22T06:31:40.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.233+0000 7fba853bb640 1 -- 172.21.15.44:0/1599674510 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba70031830 con 0x7fba800cba90 2024-04-22T06:31:40.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.233+0000 7fba853bb640 1 -- 172.21.15.44:0/1599674510 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba70031c00 con 0x7fba800cba90 2024-04-22T06:31:40.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.233+0000 7fba88621640 1 -- 172.21.15.44:0/1599674510 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba800cba90 msgr2=0x7fba80071880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:40.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.233+0000 7fba88621640 1 --2- 172.21.15.44:0/1599674510 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba800cba90 0x7fba80071880 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fba700097a0 tx=0x7fba7002b8d0 comp rx=0 tx=0).stop 2024-04-22T06:31:40.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.233+0000 7fba88621640 1 -- 172.21.15.44:0/1599674510 shutdown_connections 2024-04-22T06:31:40.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.233+0000 7fba88621640 1 --2- 172.21.15.44:0/1599674510 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba800cba90 0x7fba80071880 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:40.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.233+0000 7fba88621640 1 --2- 172.21.15.44:0/1599674510 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba800cb6b0 0x7fba8015e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:40.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.233+0000 7fba88621640 1 -- 172.21.15.44:0/1599674510 >> 172.21.15.44:0/1599674510 conn(0x7fba80074e90 msgr2=0x7fba800752a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:40.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.233+0000 7fba88621640 1 -- 172.21.15.44:0/1599674510 shutdown_connections 2024-04-22T06:31:40.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.233+0000 7fba88621640 1 -- 172.21.15.44:0/1599674510 wait complete. 2024-04-22T06:31:40.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.235+0000 7fba88621640 1 Processor -- start 2024-04-22T06:31:40.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.235+0000 7fba88621640 1 -- start start 2024-04-22T06:31:40.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.235+0000 7fba88621640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba8006c2f0 0x7fba800cc560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:40.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.235+0000 7fba88621640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba800cb6b0 0x7fba80147440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:40.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.235+0000 7fba88621640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba800cba90 0x7fba800c83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:40.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.235+0000 7fba88621640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fba8015ef10 con 0x7fba800cba90 2024-04-22T06:31:40.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.235+0000 7fba88621640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fba8015b1a0 con 0x7fba800cb6b0 2024-04-22T06:31:40.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.235+0000 7fba88621640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fba8015f090 con 0x7fba8006c2f0 2024-04-22T06:31:40.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.236+0000 7fba86bbe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba800cba90 0x7fba800c83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:40.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.236+0000 7fba85bbc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba800cb6b0 0x7fba80147440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:40.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.236+0000 7fba863bd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba8006c2f0 0x7fba800cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:40.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.236+0000 7fba86bbe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba800cba90 0x7fba800c83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53908/0 (socket says 172.21.15.44:53908) 2024-04-22T06:31:40.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.236+0000 7fba86bbe640 1 -- 172.21.15.44:0/481193874 learned_addr learned my addr 172.21.15.44:0/481193874 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:40.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.236+0000 7fba863bd640 1 -- 172.21.15.44:0/481193874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba800cb6b0 msgr2=0x7fba80147440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:40.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.236+0000 7fba863bd640 1 --2- 172.21.15.44:0/481193874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba800cb6b0 0x7fba80147440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:40.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.236+0000 7fba863bd640 1 -- 172.21.15.44:0/481193874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba800cba90 msgr2=0x7fba800c83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:40.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.236+0000 7fba863bd640 1 --2- 172.21.15.44:0/481193874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba800cba90 0x7fba800c83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:40.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.236+0000 7fba863bd640 1 -- 172.21.15.44:0/481193874 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba800c6ec0 con 0x7fba8006c2f0 2024-04-22T06:31:40.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.236+0000 7fba863bd640 1 --2- 172.21.15.44:0/481193874 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba8006c2f0 0x7fba800cc560 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fba7800efc0 tx=0x7fba7800c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:40.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.236+0000 7fba7effd640 1 -- 172.21.15.44:0/481193874 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba78018020 con 0x7fba8006c2f0 2024-04-22T06:31:40.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.236+0000 7fba7effd640 1 -- 172.21.15.44:0/481193874 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba78014680 con 0x7fba8006c2f0 2024-04-22T06:31:40.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.236+0000 7fba88621640 1 -- 172.21.15.44:0/481193874 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba800c66c0 con 0x7fba8006c2f0 2024-04-22T06:31:40.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.236+0000 7fba7effd640 1 -- 172.21.15.44:0/481193874 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba78014a70 con 0x7fba8006c2f0 2024-04-22T06:31:40.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.236+0000 7fba88621640 1 -- 172.21.15.44:0/481193874 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fba800cd0c0 con 0x7fba8006c2f0 2024-04-22T06:31:40.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.237+0000 7fba88621640 1 -- 172.21.15.44:0/481193874 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba8006c030 con 0x7fba8006c2f0 2024-04-22T06:31:40.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.238+0000 7fba7effd640 1 -- 172.21.15.44:0/481193874 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fba78005d10 con 0x7fba8006c2f0 2024-04-22T06:31:40.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.238+0000 7fba7effd640 1 --2- 172.21.15.44:0/481193874 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fba6c06c920 0x7fba6c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:40.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.238+0000 7fba7effd640 1 -- 172.21.15.44:0/481193874 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fba7808cad0 con 0x7fba8006c2f0 2024-04-22T06:31:40.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.238+0000 7fba85bbc640 1 --2- 172.21.15.44:0/481193874 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fba6c06c920 0x7fba6c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:40.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.239+0000 7fba85bbc640 1 --2- 172.21.15.44:0/481193874 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fba6c06c920 0x7fba6c06ede0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fba80149ca0 tx=0x7fba74007d50 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:40.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.242+0000 7fba7effd640 1 -- 172.21.15.44:0/481193874 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fba8006c030 con 0x7fba8006c2f0 2024-04-22T06:31:40.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.427+0000 7fba88621640 1 -- 172.21.15.44:0/481193874 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fba80068cc0 con 0x7fba8006c2f0 2024-04-22T06:31:40.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.428+0000 7fba7effd640 1 -- 172.21.15.44:0/481193874 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fba78022030 con 0x7fba8006c2f0 2024-04-22T06:31:40.426 INFO:teuthology.orchestra.run.smithi044.stdout:55834574859 2024-04-22T06:31:40.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.429+0000 7fba88621640 1 -- 172.21.15.44:0/481193874 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fba6c06c920 msgr2=0x7fba6c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:40.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.429+0000 7fba88621640 1 --2- 172.21.15.44:0/481193874 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fba6c06c920 0x7fba6c06ede0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fba80149ca0 tx=0x7fba74007d50 comp rx=0 tx=0).stop 2024-04-22T06:31:40.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.429+0000 7fba88621640 1 -- 172.21.15.44:0/481193874 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba8006c2f0 msgr2=0x7fba800cc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:40.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.429+0000 7fba88621640 1 --2- 172.21.15.44:0/481193874 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba8006c2f0 0x7fba800cc560 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fba7800efc0 tx=0x7fba7800c490 comp rx=0 tx=0).stop 2024-04-22T06:31:40.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.429+0000 7fba88621640 1 -- 172.21.15.44:0/481193874 shutdown_connections 2024-04-22T06:31:40.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.429+0000 7fba88621640 1 --2- 172.21.15.44:0/481193874 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fba6c06c920 0x7fba6c06ede0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:40.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.429+0000 7fba88621640 1 --2- 172.21.15.44:0/481193874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba800cba90 0x7fba800c83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:40.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.429+0000 7fba88621640 1 --2- 172.21.15.44:0/481193874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba800cb6b0 0x7fba80147440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:40.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.429+0000 7fba88621640 1 --2- 172.21.15.44:0/481193874 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba8006c2f0 0x7fba800cc560 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:40.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.429+0000 7fba88621640 1 -- 172.21.15.44:0/481193874 >> 172.21.15.44:0/481193874 conn(0x7fba80074e90 msgr2=0x7fba800c9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:40.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.430+0000 7fba88621640 1 -- 172.21.15.44:0/481193874 shutdown_connections 2024-04-22T06:31:40.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.430+0000 7fba88621640 1 -- 172.21.15.44:0/481193874 wait complete. 2024-04-22T06:31:40.440 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.5 2024-04-22T06:31:40.440 DEBUG:teuthology.parallel:result is None 2024-04-22T06:31:40.440 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-22T06:31:40.440 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T06:31:40.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.542+0000 7fa83eb32640 1 Processor -- start 2024-04-22T06:31:40.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.542+0000 7fa83eb32640 1 -- start start 2024-04-22T06:31:40.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.543+0000 7fa83eb32640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa838155240 0x7fa83815f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:40.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.543+0000 7fa83eb32640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa8380cc480 0x7fa8380cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:40.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.543+0000 7fa83eb32640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8380cce30 con 0x7fa8380cb320 2024-04-22T06:31:40.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.543+0000 7fa83eb32640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fa83815f610 con 0x7fa8380cc480 2024-04-22T06:31:40.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.543+0000 7fa83eb32640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa83815c060 con 0x7fa838155240 2024-04-22T06:31:40.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.543+0000 7fa83d0cf640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa8380cc480 0x7fa8380cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:40.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.543+0000 7fa83c8ce640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa838155240 0x7fa83815f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:40.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.543+0000 7fa82ffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa8380cb320 0x7fa8380d0ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.44:46280/0 (socket says 172.21.15.44:46280) 2024-04-22T06:31:40.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.543+0000 7fa82ffff640 1 -- 172.21.15.44:0/253187443 learned_addr learned my addr 172.21.15.44:0/253187443 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:40.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.544+0000 7fa83c8ce640 1 -- 172.21.15.44:0/253187443 >> v2:172.21.15.70:3300/0 conn(0x7fa8380cc480 msgr2=0x7fa8380cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:40.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.544+0000 7fa83c8ce640 1 --2- 172.21.15.44:0/253187443 >> v2:172.21.15.70:3300/0 conn(0x7fa8380cc480 0x7fa8380cc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:40.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.544+0000 7fa83c8ce640 1 -- 172.21.15.44:0/253187443 >> v1:172.21.15.70:6789/0 conn(0x7fa8380cb320 legacy=0x7fa8380d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:40.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.544+0000 7fa83c8ce640 1 -- 172.21.15.44:0/253187443 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa83815b200 con 0x7fa838155240 2024-04-22T06:31:40.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.544+0000 7fa83c8ce640 1 --2- 172.21.15.44:0/253187443 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa838155240 0x7fa83815f0d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa8340097a0 tx=0x7fa83402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f64d257cac7f9bea server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:40.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.545+0000 7fa82f7fe640 1 -- 172.21.15.44:0/253187443 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa834035020 con 0x7fa838155240 2024-04-22T06:31:40.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.545+0000 7fa82f7fe640 1 -- 172.21.15.44:0/253187443 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa834031830 con 0x7fa838155240 2024-04-22T06:31:40.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.545+0000 7fa82f7fe640 1 -- 172.21.15.44:0/253187443 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa834031c00 con 0x7fa838155240 2024-04-22T06:31:40.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.545+0000 7fa83eb32640 1 -- 172.21.15.44:0/253187443 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa838155240 msgr2=0x7fa83815f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:40.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.545+0000 7fa83eb32640 1 --2- 172.21.15.44:0/253187443 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa838155240 0x7fa83815f0d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa8340097a0 tx=0x7fa83402b8d0 comp rx=0 tx=0).stop 2024-04-22T06:31:40.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.545+0000 7fa83eb32640 1 -- 172.21.15.44:0/253187443 shutdown_connections 2024-04-22T06:31:40.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.545+0000 7fa83eb32640 1 --2- 172.21.15.44:0/253187443 >> v2:172.21.15.70:3300/0 conn(0x7fa8380cc480 0x7fa8380cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:40.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.545+0000 7fa83eb32640 1 --2- 172.21.15.44:0/253187443 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa838155240 0x7fa83815f0d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:40.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.545+0000 7fa83eb32640 1 -- 172.21.15.44:0/253187443 >> 172.21.15.44:0/253187443 conn(0x7fa8380750d0 msgr2=0x7fa8380754e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:40.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.545+0000 7fa83eb32640 1 -- 172.21.15.44:0/253187443 shutdown_connections 2024-04-22T06:31:40.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.545+0000 7fa83eb32640 1 -- 172.21.15.44:0/253187443 wait complete. 2024-04-22T06:31:40.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.546+0000 7fa83eb32640 1 Processor -- start 2024-04-22T06:31:40.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.546+0000 7fa83eb32640 1 -- start start 2024-04-22T06:31:40.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.547+0000 7fa83eb32640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8380cb320 0x7fa838158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:40.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.547+0000 7fa83eb32640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8380cc480 0x7fa83815d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:40.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.547+0000 7fa83eb32640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa838155240 0x7fa8381f0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:40.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.547+0000 7fa83eb32640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa83815baa0 con 0x7fa8380cc480 2024-04-22T06:31:40.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.547+0000 7fa83eb32640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa83815b7c0 con 0x7fa8380cb320 2024-04-22T06:31:40.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.547+0000 7fa83eb32640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa83815bd80 con 0x7fa838155240 2024-04-22T06:31:40.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.547+0000 7fa83c8ce640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8380cb320 0x7fa838158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:40.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.547+0000 7fa82ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8380cc480 0x7fa83815d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:40.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.547+0000 7fa83c8ce640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8380cb320 0x7fa838158e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:34458/0 (socket says 172.21.15.44:34458) 2024-04-22T06:31:40.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.547+0000 7fa83c8ce640 1 -- 172.21.15.44:0/3480818505 learned_addr learned my addr 172.21.15.44:0/3480818505 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:40.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.547+0000 7fa83d0cf640 1 --2- 172.21.15.44:0/3480818505 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa838155240 0x7fa8381f0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:40.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.548+0000 7fa83c8ce640 1 -- 172.21.15.44:0/3480818505 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa838155240 msgr2=0x7fa8381f0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:40.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.548+0000 7fa83c8ce640 1 --2- 172.21.15.44:0/3480818505 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa838155240 0x7fa8381f0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:40.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.548+0000 7fa83c8ce640 1 -- 172.21.15.44:0/3480818505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8380cc480 msgr2=0x7fa83815d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:40.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.548+0000 7fa83c8ce640 1 --2- 172.21.15.44:0/3480818505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8380cc480 0x7fa83815d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:40.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.548+0000 7fa83c8ce640 1 -- 172.21.15.44:0/3480818505 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa83815e770 con 0x7fa8380cb320 2024-04-22T06:31:40.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.548+0000 7fa82ffff640 1 --2- 172.21.15.44:0/3480818505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8380cc480 0x7fa83815d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:31:40.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.548+0000 7fa83c8ce640 1 --2- 172.21.15.44:0/3480818505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8380cb320 0x7fa838158e40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa834000c00 tx=0x7fa8340046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:40.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.548+0000 7fa82d7fa640 1 -- 172.21.15.44:0/3480818505 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa834040070 con 0x7fa8380cb320 2024-04-22T06:31:40.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.548+0000 7fa82d7fa640 1 -- 172.21.15.44:0/3480818505 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa83403a830 con 0x7fa8380cb320 2024-04-22T06:31:40.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.548+0000 7fa83eb32640 1 -- 172.21.15.44:0/3480818505 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8381e6db0 con 0x7fa8380cb320 2024-04-22T06:31:40.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.548+0000 7fa82d7fa640 1 -- 172.21.15.44:0/3480818505 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa834035040 con 0x7fa8380cb320 2024-04-22T06:31:40.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.548+0000 7fa83eb32640 1 -- 172.21.15.44:0/3480818505 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa838152690 con 0x7fa8380cb320 2024-04-22T06:31:40.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.549+0000 7fa83eb32640 1 -- 172.21.15.44:0/3480818505 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa838068cc0 con 0x7fa8380cb320 2024-04-22T06:31:40.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.550+0000 7fa82d7fa640 1 -- 172.21.15.44:0/3480818505 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa83404b060 con 0x7fa8380cb320 2024-04-22T06:31:40.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.550+0000 7fa82d7fa640 1 --2- 172.21.15.44:0/3480818505 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fa82406c920 0x7fa82406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:40.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.550+0000 7fa82d7fa640 1 -- 172.21.15.44:0/3480818505 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa8340ad5e0 con 0x7fa8380cb320 2024-04-22T06:31:40.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.550+0000 7fa82ffff640 1 --2- 172.21.15.44:0/3480818505 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fa82406c920 0x7fa82406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:40.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.550+0000 7fa82ffff640 1 --2- 172.21.15.44:0/3480818505 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fa82406c920 0x7fa82406ede0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa83814f1d0 tx=0x7fa828007e80 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:40.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.554+0000 7fa82d7fa640 1 -- 172.21.15.44:0/3480818505 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa834031800 con 0x7fa8380cb320 2024-04-22T06:31:40.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.736+0000 7fa83eb32640 1 -- 172.21.15.44:0/3480818505 --> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa8380cadd0 con 0x7fa82406c920 2024-04-22T06:31:40.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.751+0000 7fa82d7fa640 1 -- 172.21.15.44:0/3480818505 <== mgr.4111 v2:172.21.15.70:6832/2570466131 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332672 (secure 0 0 0) 0x7fa8380cadd0 con 0x7fa82406c920 2024-04-22T06:31:40.753 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:31:40.755 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-04-22T06:31:40.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.755+0000 7fa83eb32640 1 -- 172.21.15.44:0/3480818505 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fa82406c920 msgr2=0x7fa82406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:40.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.755+0000 7fa83eb32640 1 --2- 172.21.15.44:0/3480818505 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fa82406c920 0x7fa82406ede0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa83814f1d0 tx=0x7fa828007e80 comp rx=0 tx=0).stop 2024-04-22T06:31:40.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.755+0000 7fa83eb32640 1 -- 172.21.15.44:0/3480818505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8380cb320 msgr2=0x7fa838158e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:40.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.755+0000 7fa83eb32640 1 --2- 172.21.15.44:0/3480818505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8380cb320 0x7fa838158e40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa834000c00 tx=0x7fa8340046c0 comp rx=0 tx=0).stop 2024-04-22T06:31:40.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.755+0000 7fa83eb32640 1 -- 172.21.15.44:0/3480818505 shutdown_connections 2024-04-22T06:31:40.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.755+0000 7fa83eb32640 1 --2- 172.21.15.44:0/3480818505 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fa82406c920 0x7fa82406ede0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:40.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.755+0000 7fa83eb32640 1 --2- 172.21.15.44:0/3480818505 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa838155240 0x7fa8381f0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:40.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.755+0000 7fa83eb32640 1 --2- 172.21.15.44:0/3480818505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8380cc480 0x7fa83815d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:40.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.755+0000 7fa83eb32640 1 --2- 172.21.15.44:0/3480818505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8380cb320 0x7fa838158e40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:40.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.755+0000 7fa83eb32640 1 -- 172.21.15.44:0/3480818505 >> 172.21.15.44:0/3480818505 conn(0x7fa8380750d0 msgr2=0x7fa8380d0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:40.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.755+0000 7fa83eb32640 1 -- 172.21.15.44:0/3480818505 shutdown_connections 2024-04-22T06:31:40.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.755+0000 7fa83eb32640 1 -- 172.21.15.44:0/3480818505 wait complete. 2024-04-22T06:31:40.765 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2024-04-22T06:31:40.553336+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596414,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":246,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":193,"ondisk_log_size":193,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236120,"kb_used_data":3352,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738600,"statfs":{"total":773094113280,"available":772852326400,"internally_reserved":0,"allocated":3432448,"data_stored":2032244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1406,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002774"},"pg_stats":[{"pgid":"2.1f","version":"24'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560936+0000","last_change":"2024-04-22T06:31:06.002169+0000","last_active":"2024-04-22T06:31:12.560936+0000","last_peered":"2024-04-22T06:31:12.560936+0000","last_clean":"2024-04-22T06:31:12.560936+0000","last_became_active":"2024-04-22T06:31:06.001899+0000","last_became_peered":"2024-04-22T06:31:06.001899+0000","last_unstale":"2024-04-22T06:31:12.560936+0000","last_undegraded":"2024-04-22T06:31:12.560936+0000","last_fullsized":"2024-04-22T06:31:12.560936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:11:48.056404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559298+0000","last_change":"2024-04-22T06:31:07.025982+0000","last_active":"2024-04-22T06:31:12.559298+0000","last_peered":"2024-04-22T06:31:12.559298+0000","last_clean":"2024-04-22T06:31:12.559298+0000","last_became_active":"2024-04-22T06:31:07.025650+0000","last_became_peered":"2024-04-22T06:31:07.025650+0000","last_unstale":"2024-04-22T06:31:12.559298+0000","last_undegraded":"2024-04-22T06:31:12.559298+0000","last_fullsized":"2024-04-22T06:31:12.559298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:58:58.029898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560389+0000","last_change":"2024-04-22T06:31:11.048083+0000","last_active":"2024-04-22T06:31:12.560389+0000","last_peered":"2024-04-22T06:31:12.560389+0000","last_clean":"2024-04-22T06:31:12.560389+0000","last_became_active":"2024-04-22T06:31:11.047682+0000","last_became_peered":"2024-04-22T06:31:11.047682+0000","last_unstale":"2024-04-22T06:31:12.560389+0000","last_undegraded":"2024-04-22T06:31:12.560389+0000","last_fullsized":"2024-04-22T06:31:12.560389+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:34:30.158935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"24'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559543+0000","last_change":"2024-04-22T06:31:06.007484+0000","last_active":"2024-04-22T06:31:12.559543+0000","last_peered":"2024-04-22T06:31:12.559543+0000","last_clean":"2024-04-22T06:31:12.559543+0000","last_became_active":"2024-04-22T06:31:06.007204+0000","last_became_peered":"2024-04-22T06:31:06.007204+0000","last_unstale":"2024-04-22T06:31:12.559543+0000","last_undegraded":"2024-04-22T06:31:12.559543+0000","last_fullsized":"2024-04-22T06:31:12.559543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:39:18.905813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560596+0000","last_change":"2024-04-22T06:31:07.014658+0000","last_active":"2024-04-22T06:31:12.560596+0000","last_peered":"2024-04-22T06:31:12.560596+0000","last_clean":"2024-04-22T06:31:12.560596+0000","last_became_active":"2024-04-22T06:31:07.014283+0000","last_became_peered":"2024-04-22T06:31:07.014283+0000","last_unstale":"2024-04-22T06:31:12.560596+0000","last_undegraded":"2024-04-22T06:31:12.560596+0000","last_fullsized":"2024-04-22T06:31:12.560596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:43:01.159439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.032692+0000","last_change":"2024-04-22T06:31:11.045114+0000","last_active":"2024-04-22T06:31:12.032692+0000","last_peered":"2024-04-22T06:31:12.032692+0000","last_clean":"2024-04-22T06:31:12.032692+0000","last_became_active":"2024-04-22T06:31:11.044728+0000","last_became_peered":"2024-04-22T06:31:11.044728+0000","last_unstale":"2024-04-22T06:31:12.032692+0000","last_undegraded":"2024-04-22T06:31:12.032692+0000","last_fullsized":"2024-04-22T06:31:12.032692+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:48:19.215184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"24'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031494+0000","last_change":"2024-04-22T06:31:05.995468+0000","last_active":"2024-04-22T06:31:12.031494+0000","last_peered":"2024-04-22T06:31:12.031494+0000","last_clean":"2024-04-22T06:31:12.031494+0000","last_became_active":"2024-04-22T06:31:05.995159+0000","last_became_peered":"2024-04-22T06:31:05.995159+0000","last_unstale":"2024-04-22T06:31:12.031494+0000","last_undegraded":"2024-04-22T06:31:12.031494+0000","last_fullsized":"2024-04-22T06:31:12.031494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:01:12.223718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031966+0000","last_change":"2024-04-22T06:31:07.020024+0000","last_active":"2024-04-22T06:31:12.031966+0000","last_peered":"2024-04-22T06:31:12.031966+0000","last_clean":"2024-04-22T06:31:12.031966+0000","last_became_active":"2024-04-22T06:31:07.019490+0000","last_became_peered":"2024-04-22T06:31:07.019490+0000","last_unstale":"2024-04-22T06:31:12.031966+0000","last_undegraded":"2024-04-22T06:31:12.031966+0000","last_fullsized":"2024-04-22T06:31:12.031966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:26:41.811978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.033365+0000","last_change":"2024-04-22T06:31:11.057556+0000","last_active":"2024-04-22T06:31:12.033365+0000","last_peered":"2024-04-22T06:31:12.033365+0000","last_clean":"2024-04-22T06:31:12.033365+0000","last_became_active":"2024-04-22T06:31:11.057174+0000","last_became_peered":"2024-04-22T06:31:11.057174+0000","last_unstale":"2024-04-22T06:31:12.033365+0000","last_undegraded":"2024-04-22T06:31:12.033365+0000","last_fullsized":"2024-04-22T06:31:12.033365+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:41:19.526548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030542+0000","last_change":"2024-04-22T06:31:05.996281+0000","last_active":"2024-04-22T06:31:12.030542+0000","last_peered":"2024-04-22T06:31:12.030542+0000","last_clean":"2024-04-22T06:31:12.030542+0000","last_became_active":"2024-04-22T06:31:05.995951+0000","last_became_peered":"2024-04-22T06:31:05.995951+0000","last_unstale":"2024-04-22T06:31:12.030542+0000","last_undegraded":"2024-04-22T06:31:12.030542+0000","last_fullsized":"2024-04-22T06:31:12.030542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:01:43.732947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.032029+0000","last_change":"2024-04-22T06:31:07.017755+0000","last_active":"2024-04-22T06:31:12.032029+0000","last_peered":"2024-04-22T06:31:12.032029+0000","last_clean":"2024-04-22T06:31:12.032029+0000","last_became_active":"2024-04-22T06:31:07.017420+0000","last_became_peered":"2024-04-22T06:31:07.017420+0000","last_unstale":"2024-04-22T06:31:12.032029+0000","last_undegraded":"2024-04-22T06:31:12.032029+0000","last_fullsized":"2024-04-22T06:31:12.032029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:30:57.913020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.032802+0000","last_change":"2024-04-22T06:31:11.049497+0000","last_active":"2024-04-22T06:31:12.032802+0000","last_peered":"2024-04-22T06:31:12.032802+0000","last_clean":"2024-04-22T06:31:12.032802+0000","last_became_active":"2024-04-22T06:31:11.049079+0000","last_became_peered":"2024-04-22T06:31:11.049079+0000","last_unstale":"2024-04-22T06:31:12.032802+0000","last_undegraded":"2024-04-22T06:31:12.032802+0000","last_fullsized":"2024-04-22T06:31:12.032802+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:29:02.299691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558561+0000","last_change":"2024-04-22T06:31:07.023043+0000","last_active":"2024-04-22T06:31:12.558561+0000","last_peered":"2024-04-22T06:31:12.558561+0000","last_clean":"2024-04-22T06:31:12.558561+0000","last_became_active":"2024-04-22T06:31:07.022747+0000","last_became_peered":"2024-04-22T06:31:07.022747+0000","last_unstale":"2024-04-22T06:31:12.558561+0000","last_undegraded":"2024-04-22T06:31:12.558561+0000","last_fullsized":"2024-04-22T06:31:12.558561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:52:33.626072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"25'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:37.088829+0000","last_change":"2024-04-22T06:31:05.997936+0000","last_active":"2024-04-22T06:31:37.088829+0000","last_peered":"2024-04-22T06:31:37.088829+0000","last_clean":"2024-04-22T06:31:37.088829+0000","last_became_active":"2024-04-22T06:31:05.997618+0000","last_became_peered":"2024-04-22T06:31:05.997618+0000","last_unstale":"2024-04-22T06:31:37.088829+0000","last_undegraded":"2024-04-22T06:31:37.088829+0000","last_fullsized":"2024-04-22T06:31:37.088829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:43:00.396691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.033550+0000","last_change":"2024-04-22T06:31:11.047725+0000","last_active":"2024-04-22T06:31:12.033550+0000","last_peered":"2024-04-22T06:31:12.033550+0000","last_clean":"2024-04-22T06:31:12.033550+0000","last_became_active":"2024-04-22T06:31:11.047278+0000","last_became_peered":"2024-04-22T06:31:11.047278+0000","last_unstale":"2024-04-22T06:31:12.033550+0000","last_undegraded":"2024-04-22T06:31:12.033550+0000","last_fullsized":"2024-04-22T06:31:12.033550+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:32:56.201059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561171+0000","last_change":"2024-04-22T06:31:07.021250+0000","last_active":"2024-04-22T06:31:12.561171+0000","last_peered":"2024-04-22T06:31:12.561171+0000","last_clean":"2024-04-22T06:31:12.561171+0000","last_became_active":"2024-04-22T06:31:07.020876+0000","last_became_peered":"2024-04-22T06:31:07.020876+0000","last_unstale":"2024-04-22T06:31:12.561171+0000","last_undegraded":"2024-04-22T06:31:12.561171+0000","last_fullsized":"2024-04-22T06:31:12.561171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:21:31.210819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561332+0000","last_change":"2024-04-22T06:31:06.012775+0000","last_active":"2024-04-22T06:31:12.561332+0000","last_peered":"2024-04-22T06:31:12.561332+0000","last_clean":"2024-04-22T06:31:12.561332+0000","last_became_active":"2024-04-22T06:31:06.012474+0000","last_became_peered":"2024-04-22T06:31:06.012474+0000","last_unstale":"2024-04-22T06:31:12.561332+0000","last_undegraded":"2024-04-22T06:31:12.561332+0000","last_fullsized":"2024-04-22T06:31:12.561332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:09:48.197011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.033219+0000","last_change":"2024-04-22T06:31:11.057641+0000","last_active":"2024-04-22T06:31:12.033219+0000","last_peered":"2024-04-22T06:31:12.033219+0000","last_clean":"2024-04-22T06:31:12.033219+0000","last_became_active":"2024-04-22T06:31:11.057221+0000","last_became_peered":"2024-04-22T06:31:11.057221+0000","last_unstale":"2024-04-22T06:31:12.033219+0000","last_undegraded":"2024-04-22T06:31:12.033219+0000","last_fullsized":"2024-04-22T06:31:12.033219+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:14:57.713113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559730+0000","last_change":"2024-04-22T06:31:07.020822+0000","last_active":"2024-04-22T06:31:12.559730+0000","last_peered":"2024-04-22T06:31:12.559730+0000","last_clean":"2024-04-22T06:31:12.559730+0000","last_became_active":"2024-04-22T06:31:07.020484+0000","last_became_peered":"2024-04-22T06:31:07.020484+0000","last_unstale":"2024-04-22T06:31:12.559730+0000","last_undegraded":"2024-04-22T06:31:12.559730+0000","last_fullsized":"2024-04-22T06:31:12.559730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:18:26.522131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"25'3","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:37.096092+0000","last_change":"2024-04-22T06:31:06.000000+0000","last_active":"2024-04-22T06:31:37.096092+0000","last_peered":"2024-04-22T06:31:37.096092+0000","last_clean":"2024-04-22T06:31:37.096092+0000","last_became_active":"2024-04-22T06:31:05.999663+0000","last_became_peered":"2024-04-22T06:31:05.999663+0000","last_unstale":"2024-04-22T06:31:37.096092+0000","last_undegraded":"2024-04-22T06:31:37.096092+0000","last_fullsized":"2024-04-22T06:31:37.096092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:42.805084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560901+0000","last_change":"2024-04-22T06:31:11.057991+0000","last_active":"2024-04-22T06:31:12.560901+0000","last_peered":"2024-04-22T06:31:12.560901+0000","last_clean":"2024-04-22T06:31:12.560901+0000","last_became_active":"2024-04-22T06:31:11.057595+0000","last_became_peered":"2024-04-22T06:31:11.057595+0000","last_unstale":"2024-04-22T06:31:12.560901+0000","last_undegraded":"2024-04-22T06:31:12.560901+0000","last_fullsized":"2024-04-22T06:31:12.560901+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:17:06.366663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.033006+0000","last_change":"2024-04-22T06:31:07.017303+0000","last_active":"2024-04-22T06:31:12.033006+0000","last_peered":"2024-04-22T06:31:12.033006+0000","last_clean":"2024-04-22T06:31:12.033006+0000","last_became_active":"2024-04-22T06:31:07.016990+0000","last_became_peered":"2024-04-22T06:31:07.016990+0000","last_unstale":"2024-04-22T06:31:12.033006+0000","last_undegraded":"2024-04-22T06:31:12.033006+0000","last_fullsized":"2024-04-22T06:31:12.033006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:51:38.275472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"25'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:37.091829+0000","last_change":"2024-04-22T06:31:05.997363+0000","last_active":"2024-04-22T06:31:37.091829+0000","last_peered":"2024-04-22T06:31:37.091829+0000","last_clean":"2024-04-22T06:31:37.091829+0000","last_became_active":"2024-04-22T06:31:05.996982+0000","last_became_peered":"2024-04-22T06:31:05.996982+0000","last_unstale":"2024-04-22T06:31:37.091829+0000","last_undegraded":"2024-04-22T06:31:37.091829+0000","last_fullsized":"2024-04-22T06:31:37.091829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:23:50.052130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.033517+0000","last_change":"2024-04-22T06:31:11.048507+0000","last_active":"2024-04-22T06:31:12.033517+0000","last_peered":"2024-04-22T06:31:12.033517+0000","last_clean":"2024-04-22T06:31:12.033517+0000","last_became_active":"2024-04-22T06:31:11.047722+0000","last_became_peered":"2024-04-22T06:31:11.047722+0000","last_unstale":"2024-04-22T06:31:12.033517+0000","last_undegraded":"2024-04-22T06:31:12.033517+0000","last_fullsized":"2024-04-22T06:31:12.033517+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:04:48.732029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561139+0000","last_change":"2024-04-22T06:31:07.021095+0000","last_active":"2024-04-22T06:31:12.561139+0000","last_peered":"2024-04-22T06:31:12.561139+0000","last_clean":"2024-04-22T06:31:12.561139+0000","last_became_active":"2024-04-22T06:31:07.020814+0000","last_became_peered":"2024-04-22T06:31:07.020814+0000","last_unstale":"2024-04-22T06:31:12.561139+0000","last_undegraded":"2024-04-22T06:31:12.561139+0000","last_fullsized":"2024-04-22T06:31:12.561139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:27:08.869392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"25'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:37.089072+0000","last_change":"2024-04-22T06:31:05.997508+0000","last_active":"2024-04-22T06:31:37.089072+0000","last_peered":"2024-04-22T06:31:37.089072+0000","last_clean":"2024-04-22T06:31:37.089072+0000","last_became_active":"2024-04-22T06:31:05.997233+0000","last_became_peered":"2024-04-22T06:31:05.997233+0000","last_unstale":"2024-04-22T06:31:37.089072+0000","last_undegraded":"2024-04-22T06:31:37.089072+0000","last_fullsized":"2024-04-22T06:31:37.089072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:40:56.909549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.032801+0000","last_change":"2024-04-22T06:31:11.045424+0000","last_active":"2024-04-22T06:31:12.032801+0000","last_peered":"2024-04-22T06:31:12.032801+0000","last_clean":"2024-04-22T06:31:12.032801+0000","last_became_active":"2024-04-22T06:31:11.045061+0000","last_became_peered":"2024-04-22T06:31:11.045061+0000","last_unstale":"2024-04-22T06:31:12.032801+0000","last_undegraded":"2024-04-22T06:31:12.032801+0000","last_fullsized":"2024-04-22T06:31:12.032801+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:00:33.958921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559690+0000","last_change":"2024-04-22T06:31:07.026506+0000","last_active":"2024-04-22T06:31:12.559690+0000","last_peered":"2024-04-22T06:31:12.559690+0000","last_clean":"2024-04-22T06:31:12.559690+0000","last_became_active":"2024-04-22T06:31:07.026174+0000","last_became_peered":"2024-04-22T06:31:07.026174+0000","last_unstale":"2024-04-22T06:31:12.559690+0000","last_undegraded":"2024-04-22T06:31:12.559690+0000","last_fullsized":"2024-04-22T06:31:12.559690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:35:21.002170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560793+0000","last_change":"2024-04-22T06:31:05.997990+0000","last_active":"2024-04-22T06:31:12.560793+0000","last_peered":"2024-04-22T06:31:12.560793+0000","last_clean":"2024-04-22T06:31:12.560793+0000","last_became_active":"2024-04-22T06:31:05.997352+0000","last_became_peered":"2024-04-22T06:31:05.997352+0000","last_unstale":"2024-04-22T06:31:12.560793+0000","last_undegraded":"2024-04-22T06:31:12.560793+0000","last_fullsized":"2024-04-22T06:31:12.560793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:15:40.726807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559303+0000","last_change":"2024-04-22T06:31:11.053487+0000","last_active":"2024-04-22T06:31:12.559303+0000","last_peered":"2024-04-22T06:31:12.559303+0000","last_clean":"2024-04-22T06:31:12.559303+0000","last_became_active":"2024-04-22T06:31:11.053078+0000","last_became_peered":"2024-04-22T06:31:11.053078+0000","last_unstale":"2024-04-22T06:31:12.559303+0000","last_undegraded":"2024-04-22T06:31:12.559303+0000","last_fullsized":"2024-04-22T06:31:12.559303+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:23:16.130369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561518+0000","last_change":"2024-04-22T06:31:07.020420+0000","last_active":"2024-04-22T06:31:12.561518+0000","last_peered":"2024-04-22T06:31:12.561518+0000","last_clean":"2024-04-22T06:31:12.561518+0000","last_became_active":"2024-04-22T06:31:07.020110+0000","last_became_peered":"2024-04-22T06:31:07.020110+0000","last_unstale":"2024-04-22T06:31:12.561518+0000","last_undegraded":"2024-04-22T06:31:12.561518+0000","last_fullsized":"2024-04-22T06:31:12.561518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:44:23.997057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"25'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031781+0000","last_change":"2024-04-22T06:31:05.997357+0000","last_active":"2024-04-22T06:31:12.031781+0000","last_peered":"2024-04-22T06:31:12.031781+0000","last_clean":"2024-04-22T06:31:12.031781+0000","last_became_active":"2024-04-22T06:31:05.996913+0000","last_became_peered":"2024-04-22T06:31:05.996913+0000","last_unstale":"2024-04-22T06:31:12.031781+0000","last_undegraded":"2024-04-22T06:31:12.031781+0000","last_fullsized":"2024-04-22T06:31:12.031781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:50:51.880199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560195+0000","last_change":"2024-04-22T06:31:11.059735+0000","last_active":"2024-04-22T06:31:12.560195+0000","last_peered":"2024-04-22T06:31:12.560195+0000","last_clean":"2024-04-22T06:31:12.560195+0000","last_became_active":"2024-04-22T06:31:11.059325+0000","last_became_peered":"2024-04-22T06:31:11.059325+0000","last_unstale":"2024-04-22T06:31:12.560195+0000","last_undegraded":"2024-04-22T06:31:12.560195+0000","last_fullsized":"2024-04-22T06:31:12.560195+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:59:52.881783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.032101+0000","last_change":"2024-04-22T06:31:07.026576+0000","last_active":"2024-04-22T06:31:12.032101+0000","last_peered":"2024-04-22T06:31:12.032101+0000","last_clean":"2024-04-22T06:31:12.032101+0000","last_became_active":"2024-04-22T06:31:07.026057+0000","last_became_peered":"2024-04-22T06:31:07.026057+0000","last_unstale":"2024-04-22T06:31:12.032101+0000","last_undegraded":"2024-04-22T06:31:12.032101+0000","last_fullsized":"2024-04-22T06:31:12.032101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:49:32.525918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561588+0000","last_change":"2024-04-22T06:31:06.009043+0000","last_active":"2024-04-22T06:31:12.561588+0000","last_peered":"2024-04-22T06:31:12.561588+0000","last_clean":"2024-04-22T06:31:12.561588+0000","last_became_active":"2024-04-22T06:31:06.008752+0000","last_became_peered":"2024-04-22T06:31:06.008752+0000","last_unstale":"2024-04-22T06:31:12.561588+0000","last_undegraded":"2024-04-22T06:31:12.561588+0000","last_fullsized":"2024-04-22T06:31:12.561588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:52:34.706288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561059+0000","last_change":"2024-04-22T06:31:11.057688+0000","last_active":"2024-04-22T06:31:12.561059+0000","last_peered":"2024-04-22T06:31:12.561059+0000","last_clean":"2024-04-22T06:31:12.561059+0000","last_became_active":"2024-04-22T06:31:11.057354+0000","last_became_peered":"2024-04-22T06:31:11.057354+0000","last_unstale":"2024-04-22T06:31:12.561059+0000","last_undegraded":"2024-04-22T06:31:12.561059+0000","last_fullsized":"2024-04-22T06:31:12.561059+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:08:36.080352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.033017+0000","last_change":"2024-04-22T06:31:07.028506+0000","last_active":"2024-04-22T06:31:12.033017+0000","last_peered":"2024-04-22T06:31:12.033017+0000","last_clean":"2024-04-22T06:31:12.033017+0000","last_became_active":"2024-04-22T06:31:07.028183+0000","last_became_peered":"2024-04-22T06:31:07.028183+0000","last_unstale":"2024-04-22T06:31:12.033017+0000","last_undegraded":"2024-04-22T06:31:12.033017+0000","last_fullsized":"2024-04-22T06:31:12.033017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:38:45.965934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031683+0000","last_change":"2024-04-22T06:31:06.002052+0000","last_active":"2024-04-22T06:31:12.031683+0000","last_peered":"2024-04-22T06:31:12.031683+0000","last_clean":"2024-04-22T06:31:12.031683+0000","last_became_active":"2024-04-22T06:31:06.000645+0000","last_became_peered":"2024-04-22T06:31:06.000645+0000","last_unstale":"2024-04-22T06:31:12.031683+0000","last_undegraded":"2024-04-22T06:31:12.031683+0000","last_fullsized":"2024-04-22T06:31:12.031683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:55:38.982563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561693+0000","last_change":"2024-04-22T06:31:11.048333+0000","last_active":"2024-04-22T06:31:12.561693+0000","last_peered":"2024-04-22T06:31:12.561693+0000","last_clean":"2024-04-22T06:31:12.561693+0000","last_became_active":"2024-04-22T06:31:11.047913+0000","last_became_peered":"2024-04-22T06:31:11.047913+0000","last_unstale":"2024-04-22T06:31:12.561693+0000","last_undegraded":"2024-04-22T06:31:12.561693+0000","last_fullsized":"2024-04-22T06:31:12.561693+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:11:52.622436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561265+0000","last_change":"2024-04-22T06:31:07.017430+0000","last_active":"2024-04-22T06:31:12.561265+0000","last_peered":"2024-04-22T06:31:12.561265+0000","last_clean":"2024-04-22T06:31:12.561265+0000","last_became_active":"2024-04-22T06:31:07.017112+0000","last_became_peered":"2024-04-22T06:31:07.017112+0000","last_unstale":"2024-04-22T06:31:12.561265+0000","last_undegraded":"2024-04-22T06:31:12.561265+0000","last_fullsized":"2024-04-22T06:31:12.561265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:52:18.098301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"25'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558899+0000","last_change":"2024-04-22T06:31:06.002202+0000","last_active":"2024-04-22T06:31:12.558899+0000","last_peered":"2024-04-22T06:31:12.558899+0000","last_clean":"2024-04-22T06:31:12.558899+0000","last_became_active":"2024-04-22T06:31:06.001916+0000","last_became_peered":"2024-04-22T06:31:06.001916+0000","last_unstale":"2024-04-22T06:31:12.558899+0000","last_undegraded":"2024-04-22T06:31:12.558899+0000","last_fullsized":"2024-04-22T06:31:12.558899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:35:52.485268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560894+0000","last_change":"2024-04-22T06:31:11.054576+0000","last_active":"2024-04-22T06:31:12.560894+0000","last_peered":"2024-04-22T06:31:12.560894+0000","last_clean":"2024-04-22T06:31:12.560894+0000","last_became_active":"2024-04-22T06:31:11.054158+0000","last_became_peered":"2024-04-22T06:31:11.054158+0000","last_unstale":"2024-04-22T06:31:12.560894+0000","last_undegraded":"2024-04-22T06:31:12.560894+0000","last_fullsized":"2024-04-22T06:31:12.560894+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:48:50.809440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558837+0000","last_change":"2024-04-22T06:31:07.023064+0000","last_active":"2024-04-22T06:31:12.558837+0000","last_peered":"2024-04-22T06:31:12.558837+0000","last_clean":"2024-04-22T06:31:12.558837+0000","last_became_active":"2024-04-22T06:31:07.022753+0000","last_became_peered":"2024-04-22T06:31:07.022753+0000","last_unstale":"2024-04-22T06:31:12.558837+0000","last_undegraded":"2024-04-22T06:31:12.558837+0000","last_fullsized":"2024-04-22T06:31:12.558837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:25:38.039689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"25'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.029986+0000","last_change":"2024-04-22T06:31:05.998384+0000","last_active":"2024-04-22T06:31:12.029986+0000","last_peered":"2024-04-22T06:31:12.029986+0000","last_clean":"2024-04-22T06:31:12.029986+0000","last_became_active":"2024-04-22T06:31:05.997761+0000","last_became_peered":"2024-04-22T06:31:05.997761+0000","last_unstale":"2024-04-22T06:31:12.029986+0000","last_undegraded":"2024-04-22T06:31:12.029986+0000","last_fullsized":"2024-04-22T06:31:12.029986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:07:12.014505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561016+0000","last_change":"2024-04-22T06:31:11.060877+0000","last_active":"2024-04-22T06:31:12.561016+0000","last_peered":"2024-04-22T06:31:12.561016+0000","last_clean":"2024-04-22T06:31:12.561016+0000","last_became_active":"2024-04-22T06:31:11.060473+0000","last_became_peered":"2024-04-22T06:31:11.060473+0000","last_unstale":"2024-04-22T06:31:12.561016+0000","last_undegraded":"2024-04-22T06:31:12.561016+0000","last_fullsized":"2024-04-22T06:31:12.561016+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:47:18.456791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558817+0000","last_change":"2024-04-22T06:31:07.026139+0000","last_active":"2024-04-22T06:31:12.558817+0000","last_peered":"2024-04-22T06:31:12.558817+0000","last_clean":"2024-04-22T06:31:12.558817+0000","last_became_active":"2024-04-22T06:31:07.025826+0000","last_became_peered":"2024-04-22T06:31:07.025826+0000","last_unstale":"2024-04-22T06:31:12.558817+0000","last_undegraded":"2024-04-22T06:31:12.558817+0000","last_fullsized":"2024-04-22T06:31:12.558817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:58:48.200581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"25'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:37.095920+0000","last_change":"2024-04-22T06:31:06.001815+0000","last_active":"2024-04-22T06:31:37.095920+0000","last_peered":"2024-04-22T06:31:37.095920+0000","last_clean":"2024-04-22T06:31:37.095920+0000","last_became_active":"2024-04-22T06:31:06.000629+0000","last_became_peered":"2024-04-22T06:31:06.000629+0000","last_unstale":"2024-04-22T06:31:37.095920+0000","last_undegraded":"2024-04-22T06:31:37.095920+0000","last_fullsized":"2024-04-22T06:31:37.095920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:49:31.100969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030036+0000","last_change":"2024-04-22T06:30:56.214982+0000","last_active":"2024-04-22T06:31:12.030036+0000","last_peered":"2024-04-22T06:31:12.030036+0000","last_clean":"2024-04-22T06:31:12.030036+0000","last_became_active":"2024-04-22T06:30:56.214024+0000","last_became_peered":"2024-04-22T06:30:56.214024+0000","last_unstale":"2024-04-22T06:31:12.030036+0000","last_undegraded":"2024-04-22T06:31:12.030036+0000","last_fullsized":"2024-04-22T06:31:12.030036+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:30:54.947156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:30:54.947156+0000","last_clean_scrub_stamp":"2024-04-22T06:30:54.947156+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:32:37.920238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558994+0000","last_change":"2024-04-22T06:31:11.040152+0000","last_active":"2024-04-22T06:31:12.558994+0000","last_peered":"2024-04-22T06:31:12.558994+0000","last_clean":"2024-04-22T06:31:12.558994+0000","last_became_active":"2024-04-22T06:31:11.039749+0000","last_became_peered":"2024-04-22T06:31:11.039749+0000","last_unstale":"2024-04-22T06:31:12.558994+0000","last_undegraded":"2024-04-22T06:31:12.558994+0000","last_fullsized":"2024-04-22T06:31:12.558994+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:31:23.103876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031283+0000","last_change":"2024-04-22T06:31:07.011062+0000","last_active":"2024-04-22T06:31:12.031283+0000","last_peered":"2024-04-22T06:31:12.031283+0000","last_clean":"2024-04-22T06:31:12.031283+0000","last_became_active":"2024-04-22T06:31:07.010755+0000","last_became_peered":"2024-04-22T06:31:07.010755+0000","last_unstale":"2024-04-22T06:31:12.031283+0000","last_undegraded":"2024-04-22T06:31:12.031283+0000","last_fullsized":"2024-04-22T06:31:12.031283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:57:01.373036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560069+0000","last_change":"2024-04-22T06:31:05.998083+0000","last_active":"2024-04-22T06:31:12.560069+0000","last_peered":"2024-04-22T06:31:12.560069+0000","last_clean":"2024-04-22T06:31:12.560069+0000","last_became_active":"2024-04-22T06:31:05.997777+0000","last_became_peered":"2024-04-22T06:31:05.997777+0000","last_unstale":"2024-04-22T06:31:12.560069+0000","last_undegraded":"2024-04-22T06:31:12.560069+0000","last_fullsized":"2024-04-22T06:31:12.560069+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:15:13.171198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558666+0000","last_change":"2024-04-22T06:31:11.049607+0000","last_active":"2024-04-22T06:31:12.558666+0000","last_peered":"2024-04-22T06:31:12.558666+0000","last_clean":"2024-04-22T06:31:12.558666+0000","last_became_active":"2024-04-22T06:31:11.049156+0000","last_became_peered":"2024-04-22T06:31:11.049156+0000","last_unstale":"2024-04-22T06:31:12.558666+0000","last_undegraded":"2024-04-22T06:31:12.558666+0000","last_fullsized":"2024-04-22T06:31:12.558666+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:15:48.599278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.032032+0000","last_change":"2024-04-22T06:31:07.026306+0000","last_active":"2024-04-22T06:31:12.032032+0000","last_peered":"2024-04-22T06:31:12.032032+0000","last_clean":"2024-04-22T06:31:12.032032+0000","last_became_active":"2024-04-22T06:31:07.025987+0000","last_became_peered":"2024-04-22T06:31:07.025987+0000","last_unstale":"2024-04-22T06:31:12.032032+0000","last_undegraded":"2024-04-22T06:31:12.032032+0000","last_fullsized":"2024-04-22T06:31:12.032032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:50:31.803180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560942+0000","last_change":"2024-04-22T06:31:06.003115+0000","last_active":"2024-04-22T06:31:12.560942+0000","last_peered":"2024-04-22T06:31:12.560942+0000","last_clean":"2024-04-22T06:31:12.560942+0000","last_became_active":"2024-04-22T06:31:06.001945+0000","last_became_peered":"2024-04-22T06:31:06.001945+0000","last_unstale":"2024-04-22T06:31:12.560942+0000","last_undegraded":"2024-04-22T06:31:12.560942+0000","last_fullsized":"2024-04-22T06:31:12.560942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:59:25.770504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561105+0000","last_change":"2024-04-22T06:31:11.058841+0000","last_active":"2024-04-22T06:31:12.561105+0000","last_peered":"2024-04-22T06:31:12.561105+0000","last_clean":"2024-04-22T06:31:12.561105+0000","last_became_active":"2024-04-22T06:31:11.058407+0000","last_became_peered":"2024-04-22T06:31:11.058407+0000","last_unstale":"2024-04-22T06:31:12.561105+0000","last_undegraded":"2024-04-22T06:31:12.561105+0000","last_fullsized":"2024-04-22T06:31:12.561105+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:30:20.432210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031032+0000","last_change":"2024-04-22T06:31:07.022314+0000","last_active":"2024-04-22T06:31:12.031032+0000","last_peered":"2024-04-22T06:31:12.031032+0000","last_clean":"2024-04-22T06:31:12.031032+0000","last_became_active":"2024-04-22T06:31:07.021497+0000","last_became_peered":"2024-04-22T06:31:07.021497+0000","last_unstale":"2024-04-22T06:31:12.031032+0000","last_undegraded":"2024-04-22T06:31:12.031032+0000","last_fullsized":"2024-04-22T06:31:12.031032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:16:25.960359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559348+0000","last_change":"2024-04-22T06:31:06.003805+0000","last_active":"2024-04-22T06:31:12.559348+0000","last_peered":"2024-04-22T06:31:12.559348+0000","last_clean":"2024-04-22T06:31:12.559348+0000","last_became_active":"2024-04-22T06:31:06.002735+0000","last_became_peered":"2024-04-22T06:31:06.002735+0000","last_unstale":"2024-04-22T06:31:12.559348+0000","last_undegraded":"2024-04-22T06:31:12.559348+0000","last_fullsized":"2024-04-22T06:31:12.559348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:41:16.155628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030380+0000","last_change":"2024-04-22T06:31:11.043003+0000","last_active":"2024-04-22T06:31:12.030380+0000","last_peered":"2024-04-22T06:31:12.030380+0000","last_clean":"2024-04-22T06:31:12.030380+0000","last_became_active":"2024-04-22T06:31:11.042599+0000","last_became_peered":"2024-04-22T06:31:11.042599+0000","last_unstale":"2024-04-22T06:31:12.030380+0000","last_undegraded":"2024-04-22T06:31:12.030380+0000","last_fullsized":"2024-04-22T06:31:12.030380+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:28:58.949214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030520+0000","last_change":"2024-04-22T06:31:07.021258+0000","last_active":"2024-04-22T06:31:12.030520+0000","last_peered":"2024-04-22T06:31:12.030520+0000","last_clean":"2024-04-22T06:31:12.030520+0000","last_became_active":"2024-04-22T06:31:07.020976+0000","last_became_peered":"2024-04-22T06:31:07.020976+0000","last_unstale":"2024-04-22T06:31:12.030520+0000","last_undegraded":"2024-04-22T06:31:12.030520+0000","last_fullsized":"2024-04-22T06:31:12.030520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:19:33.377626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"24'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.033565+0000","last_change":"2024-04-22T06:31:06.002751+0000","last_active":"2024-04-22T06:31:12.033565+0000","last_peered":"2024-04-22T06:31:12.033565+0000","last_clean":"2024-04-22T06:31:12.033565+0000","last_became_active":"2024-04-22T06:31:06.002489+0000","last_became_peered":"2024-04-22T06:31:06.002489+0000","last_unstale":"2024-04-22T06:31:12.033565+0000","last_undegraded":"2024-04-22T06:31:12.033565+0000","last_fullsized":"2024-04-22T06:31:12.033565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:20:05.937529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.032720+0000","last_change":"2024-04-22T06:31:11.048954+0000","last_active":"2024-04-22T06:31:12.032720+0000","last_peered":"2024-04-22T06:31:12.032720+0000","last_clean":"2024-04-22T06:31:12.032720+0000","last_became_active":"2024-04-22T06:31:11.048214+0000","last_became_peered":"2024-04-22T06:31:11.048214+0000","last_unstale":"2024-04-22T06:31:12.032720+0000","last_undegraded":"2024-04-22T06:31:12.032720+0000","last_fullsized":"2024-04-22T06:31:12.032720+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:13:21.903203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030103+0000","last_change":"2024-04-22T06:31:07.021258+0000","last_active":"2024-04-22T06:31:12.030103+0000","last_peered":"2024-04-22T06:31:12.030103+0000","last_clean":"2024-04-22T06:31:12.030103+0000","last_became_active":"2024-04-22T06:31:07.021036+0000","last_became_peered":"2024-04-22T06:31:07.021036+0000","last_unstale":"2024-04-22T06:31:12.030103+0000","last_undegraded":"2024-04-22T06:31:12.030103+0000","last_fullsized":"2024-04-22T06:31:12.030103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:51:39.990228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"25'10","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:37.093513+0000","last_change":"2024-04-22T06:31:05.998270+0000","last_active":"2024-04-22T06:31:37.093513+0000","last_peered":"2024-04-22T06:31:37.093513+0000","last_clean":"2024-04-22T06:31:37.093513+0000","last_became_active":"2024-04-22T06:31:05.997925+0000","last_became_peered":"2024-04-22T06:31:05.997925+0000","last_unstale":"2024-04-22T06:31:37.093513+0000","last_undegraded":"2024-04-22T06:31:37.093513+0000","last_fullsized":"2024-04-22T06:31:37.093513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:49:44.167932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560845+0000","last_change":"2024-04-22T06:31:11.051106+0000","last_active":"2024-04-22T06:31:12.560845+0000","last_peered":"2024-04-22T06:31:12.560845+0000","last_clean":"2024-04-22T06:31:12.560845+0000","last_became_active":"2024-04-22T06:31:11.050721+0000","last_became_peered":"2024-04-22T06:31:11.050721+0000","last_unstale":"2024-04-22T06:31:12.560845+0000","last_undegraded":"2024-04-22T06:31:12.560845+0000","last_fullsized":"2024-04-22T06:31:12.560845+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:58:43.521850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558584+0000","last_change":"2024-04-22T06:31:07.020281+0000","last_active":"2024-04-22T06:31:12.558584+0000","last_peered":"2024-04-22T06:31:12.558584+0000","last_clean":"2024-04-22T06:31:12.558584+0000","last_became_active":"2024-04-22T06:31:07.020012+0000","last_became_peered":"2024-04-22T06:31:07.020012+0000","last_unstale":"2024-04-22T06:31:12.558584+0000","last_undegraded":"2024-04-22T06:31:12.558584+0000","last_fullsized":"2024-04-22T06:31:12.558584+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:20:40.301631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558743+0000","last_change":"2024-04-22T06:31:05.999918+0000","last_active":"2024-04-22T06:31:12.558743+0000","last_peered":"2024-04-22T06:31:12.558743+0000","last_clean":"2024-04-22T06:31:12.558743+0000","last_became_active":"2024-04-22T06:31:05.999594+0000","last_became_peered":"2024-04-22T06:31:05.999594+0000","last_unstale":"2024-04-22T06:31:12.558743+0000","last_undegraded":"2024-04-22T06:31:12.558743+0000","last_fullsized":"2024-04-22T06:31:12.558743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:15:21.376196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559535+0000","last_change":"2024-04-22T06:31:11.055940+0000","last_active":"2024-04-22T06:31:12.559535+0000","last_peered":"2024-04-22T06:31:12.559535+0000","last_clean":"2024-04-22T06:31:12.559535+0000","last_became_active":"2024-04-22T06:31:11.054873+0000","last_became_peered":"2024-04-22T06:31:11.054873+0000","last_unstale":"2024-04-22T06:31:12.559535+0000","last_undegraded":"2024-04-22T06:31:12.559535+0000","last_fullsized":"2024-04-22T06:31:12.559535+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:13:51.061199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030857+0000","last_change":"2024-04-22T06:31:07.021839+0000","last_active":"2024-04-22T06:31:12.030857+0000","last_peered":"2024-04-22T06:31:12.030857+0000","last_clean":"2024-04-22T06:31:12.030857+0000","last_became_active":"2024-04-22T06:31:07.021585+0000","last_became_peered":"2024-04-22T06:31:07.021585+0000","last_unstale":"2024-04-22T06:31:12.030857+0000","last_undegraded":"2024-04-22T06:31:12.030857+0000","last_fullsized":"2024-04-22T06:31:12.030857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:55:15.275802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"25'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:37.093627+0000","last_change":"2024-04-22T06:31:06.004058+0000","last_active":"2024-04-22T06:31:37.093627+0000","last_peered":"2024-04-22T06:31:37.093627+0000","last_clean":"2024-04-22T06:31:37.093627+0000","last_became_active":"2024-04-22T06:31:06.003791+0000","last_became_peered":"2024-04-22T06:31:06.003791+0000","last_unstale":"2024-04-22T06:31:37.093627+0000","last_undegraded":"2024-04-22T06:31:37.093627+0000","last_fullsized":"2024-04-22T06:31:37.093627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:03:08.913113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559058+0000","last_change":"2024-04-22T06:31:11.054245+0000","last_active":"2024-04-22T06:31:12.559058+0000","last_peered":"2024-04-22T06:31:12.559058+0000","last_clean":"2024-04-22T06:31:12.559058+0000","last_became_active":"2024-04-22T06:31:11.053551+0000","last_became_peered":"2024-04-22T06:31:11.053551+0000","last_unstale":"2024-04-22T06:31:12.559058+0000","last_undegraded":"2024-04-22T06:31:12.559058+0000","last_fullsized":"2024-04-22T06:31:12.559058+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:34:06.312700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559478+0000","last_change":"2024-04-22T06:31:07.023273+0000","last_active":"2024-04-22T06:31:12.559478+0000","last_peered":"2024-04-22T06:31:12.559478+0000","last_clean":"2024-04-22T06:31:12.559478+0000","last_became_active":"2024-04-22T06:31:07.022978+0000","last_became_peered":"2024-04-22T06:31:07.022978+0000","last_unstale":"2024-04-22T06:31:12.559478+0000","last_undegraded":"2024-04-22T06:31:12.559478+0000","last_fullsized":"2024-04-22T06:31:12.559478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:10:50.192298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"25'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:37.091741+0000","last_change":"2024-04-22T06:31:06.005510+0000","last_active":"2024-04-22T06:31:37.091741+0000","last_peered":"2024-04-22T06:31:37.091741+0000","last_clean":"2024-04-22T06:31:37.091741+0000","last_became_active":"2024-04-22T06:31:06.005047+0000","last_became_peered":"2024-04-22T06:31:06.005047+0000","last_unstale":"2024-04-22T06:31:37.091741+0000","last_undegraded":"2024-04-22T06:31:37.091741+0000","last_fullsized":"2024-04-22T06:31:37.091741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:25:51.227631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.032100+0000","last_change":"2024-04-22T06:31:11.046209+0000","last_active":"2024-04-22T06:31:12.032100+0000","last_peered":"2024-04-22T06:31:12.032100+0000","last_clean":"2024-04-22T06:31:12.032100+0000","last_became_active":"2024-04-22T06:31:11.045339+0000","last_became_peered":"2024-04-22T06:31:11.045339+0000","last_unstale":"2024-04-22T06:31:12.032100+0000","last_undegraded":"2024-04-22T06:31:12.032100+0000","last_fullsized":"2024-04-22T06:31:12.032100+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:39:16.659010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031172+0000","last_change":"2024-04-22T06:31:07.025901+0000","last_active":"2024-04-22T06:31:12.031172+0000","last_peered":"2024-04-22T06:31:12.031172+0000","last_clean":"2024-04-22T06:31:12.031172+0000","last_became_active":"2024-04-22T06:31:07.025590+0000","last_became_peered":"2024-04-22T06:31:07.025590+0000","last_unstale":"2024-04-22T06:31:12.031172+0000","last_undegraded":"2024-04-22T06:31:12.031172+0000","last_fullsized":"2024-04-22T06:31:12.031172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:00:26.184054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'11","reported_seq":33,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:37.094322+0000","last_change":"2024-04-22T06:31:06.004609+0000","last_active":"2024-04-22T06:31:37.094322+0000","last_peered":"2024-04-22T06:31:37.094322+0000","last_clean":"2024-04-22T06:31:37.094322+0000","last_became_active":"2024-04-22T06:31:06.004319+0000","last_became_peered":"2024-04-22T06:31:06.004319+0000","last_unstale":"2024-04-22T06:31:37.094322+0000","last_undegraded":"2024-04-22T06:31:37.094322+0000","last_fullsized":"2024-04-22T06:31:37.094322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:32:47.198906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561266+0000","last_change":"2024-04-22T06:31:11.054706+0000","last_active":"2024-04-22T06:31:12.561266+0000","last_peered":"2024-04-22T06:31:12.561266+0000","last_clean":"2024-04-22T06:31:12.561266+0000","last_became_active":"2024-04-22T06:31:11.054307+0000","last_became_peered":"2024-04-22T06:31:11.054307+0000","last_unstale":"2024-04-22T06:31:12.561266+0000","last_undegraded":"2024-04-22T06:31:12.561266+0000","last_fullsized":"2024-04-22T06:31:12.561266+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:05:02.465114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.033403+0000","last_change":"2024-04-22T06:31:07.020739+0000","last_active":"2024-04-22T06:31:12.033403+0000","last_peered":"2024-04-22T06:31:12.033403+0000","last_clean":"2024-04-22T06:31:12.033403+0000","last_became_active":"2024-04-22T06:31:07.020070+0000","last_became_peered":"2024-04-22T06:31:07.020070+0000","last_unstale":"2024-04-22T06:31:12.033403+0000","last_undegraded":"2024-04-22T06:31:12.033403+0000","last_fullsized":"2024-04-22T06:31:12.033403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:50:22.102753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"25'4","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560870+0000","last_change":"2024-04-22T06:31:06.003733+0000","last_active":"2024-04-22T06:31:12.560870+0000","last_peered":"2024-04-22T06:31:12.560870+0000","last_clean":"2024-04-22T06:31:12.560870+0000","last_became_active":"2024-04-22T06:31:06.003465+0000","last_became_peered":"2024-04-22T06:31:06.003465+0000","last_unstale":"2024-04-22T06:31:12.560870+0000","last_undegraded":"2024-04-22T06:31:12.560870+0000","last_fullsized":"2024-04-22T06:31:12.560870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:19:36.023311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.032875+0000","last_change":"2024-04-22T06:31:11.058041+0000","last_active":"2024-04-22T06:31:12.032875+0000","last_peered":"2024-04-22T06:31:12.032875+0000","last_clean":"2024-04-22T06:31:12.032875+0000","last_became_active":"2024-04-22T06:31:11.057243+0000","last_became_peered":"2024-04-22T06:31:11.057243+0000","last_unstale":"2024-04-22T06:31:12.032875+0000","last_undegraded":"2024-04-22T06:31:12.032875+0000","last_fullsized":"2024-04-22T06:31:12.032875+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:49:06.781140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560767+0000","last_change":"2024-04-22T06:31:07.015369+0000","last_active":"2024-04-22T06:31:12.560767+0000","last_peered":"2024-04-22T06:31:12.560767+0000","last_clean":"2024-04-22T06:31:12.560767+0000","last_became_active":"2024-04-22T06:31:07.015088+0000","last_became_peered":"2024-04-22T06:31:07.015088+0000","last_unstale":"2024-04-22T06:31:12.560767+0000","last_undegraded":"2024-04-22T06:31:12.560767+0000","last_fullsized":"2024-04-22T06:31:12.560767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:10:09.449507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"25'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:37.089606+0000","last_change":"2024-04-22T06:31:05.999636+0000","last_active":"2024-04-22T06:31:37.089606+0000","last_peered":"2024-04-22T06:31:37.089606+0000","last_clean":"2024-04-22T06:31:37.089606+0000","last_became_active":"2024-04-22T06:31:05.999329+0000","last_became_peered":"2024-04-22T06:31:05.999329+0000","last_unstale":"2024-04-22T06:31:37.089606+0000","last_undegraded":"2024-04-22T06:31:37.089606+0000","last_fullsized":"2024-04-22T06:31:37.089606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:09:06.670827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559595+0000","last_change":"2024-04-22T06:31:11.053530+0000","last_active":"2024-04-22T06:31:12.559595+0000","last_peered":"2024-04-22T06:31:12.559595+0000","last_clean":"2024-04-22T06:31:12.559595+0000","last_became_active":"2024-04-22T06:31:11.053100+0000","last_became_peered":"2024-04-22T06:31:11.053100+0000","last_unstale":"2024-04-22T06:31:12.559595+0000","last_undegraded":"2024-04-22T06:31:12.559595+0000","last_fullsized":"2024-04-22T06:31:12.559595+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:57:16.506840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030657+0000","last_change":"2024-04-22T06:31:07.015838+0000","last_active":"2024-04-22T06:31:12.030657+0000","last_peered":"2024-04-22T06:31:12.030657+0000","last_clean":"2024-04-22T06:31:12.030657+0000","last_became_active":"2024-04-22T06:31:07.015545+0000","last_became_peered":"2024-04-22T06:31:07.015545+0000","last_unstale":"2024-04-22T06:31:12.030657+0000","last_undegraded":"2024-04-22T06:31:12.030657+0000","last_fullsized":"2024-04-22T06:31:12.030657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:43:54.251061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"25'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:37.089896+0000","last_change":"2024-04-22T06:31:06.005442+0000","last_active":"2024-04-22T06:31:37.089896+0000","last_peered":"2024-04-22T06:31:37.089896+0000","last_clean":"2024-04-22T06:31:37.089896+0000","last_became_active":"2024-04-22T06:31:06.005156+0000","last_became_peered":"2024-04-22T06:31:06.005156+0000","last_unstale":"2024-04-22T06:31:37.089896+0000","last_undegraded":"2024-04-22T06:31:37.089896+0000","last_fullsized":"2024-04-22T06:31:37.089896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:52:32.138519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560401+0000","last_change":"2024-04-22T06:31:11.046885+0000","last_active":"2024-04-22T06:31:12.560401+0000","last_peered":"2024-04-22T06:31:12.560401+0000","last_clean":"2024-04-22T06:31:12.560401+0000","last_became_active":"2024-04-22T06:31:11.046444+0000","last_became_peered":"2024-04-22T06:31:11.046444+0000","last_unstale":"2024-04-22T06:31:12.560401+0000","last_undegraded":"2024-04-22T06:31:12.560401+0000","last_fullsized":"2024-04-22T06:31:12.560401+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:17:54.523416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560687+0000","last_change":"2024-04-22T06:31:11.061784+0000","last_active":"2024-04-22T06:31:12.560687+0000","last_peered":"2024-04-22T06:31:12.560687+0000","last_clean":"2024-04-22T06:31:12.560687+0000","last_became_active":"2024-04-22T06:31:11.061382+0000","last_became_peered":"2024-04-22T06:31:11.061382+0000","last_unstale":"2024-04-22T06:31:12.560687+0000","last_undegraded":"2024-04-22T06:31:12.560687+0000","last_fullsized":"2024-04-22T06:31:12.560687+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:12:53.829206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560353+0000","last_change":"2024-04-22T06:31:07.023542+0000","last_active":"2024-04-22T06:31:12.560353+0000","last_peered":"2024-04-22T06:31:12.560353+0000","last_clean":"2024-04-22T06:31:12.560353+0000","last_became_active":"2024-04-22T06:31:07.023240+0000","last_became_peered":"2024-04-22T06:31:07.023240+0000","last_unstale":"2024-04-22T06:31:12.560353+0000","last_undegraded":"2024-04-22T06:31:12.560353+0000","last_fullsized":"2024-04-22T06:31:12.560353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:15:34.727195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030715+0000","last_change":"2024-04-22T06:31:06.003080+0000","last_active":"2024-04-22T06:31:12.030715+0000","last_peered":"2024-04-22T06:31:12.030715+0000","last_clean":"2024-04-22T06:31:12.030715+0000","last_became_active":"2024-04-22T06:31:06.002725+0000","last_became_peered":"2024-04-22T06:31:06.002725+0000","last_unstale":"2024-04-22T06:31:12.030715+0000","last_undegraded":"2024-04-22T06:31:12.030715+0000","last_fullsized":"2024-04-22T06:31:12.030715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:46:38.249137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559100+0000","last_change":"2024-04-22T06:31:11.047782+0000","last_active":"2024-04-22T06:31:12.559100+0000","last_peered":"2024-04-22T06:31:12.559100+0000","last_clean":"2024-04-22T06:31:12.559100+0000","last_became_active":"2024-04-22T06:31:11.047407+0000","last_became_peered":"2024-04-22T06:31:11.047407+0000","last_unstale":"2024-04-22T06:31:12.559100+0000","last_undegraded":"2024-04-22T06:31:12.559100+0000","last_fullsized":"2024-04-22T06:31:12.559100+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:48:05.416994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559643+0000","last_change":"2024-04-22T06:31:07.025799+0000","last_active":"2024-04-22T06:31:12.559643+0000","last_peered":"2024-04-22T06:31:12.559643+0000","last_clean":"2024-04-22T06:31:12.559643+0000","last_became_active":"2024-04-22T06:31:07.025420+0000","last_became_peered":"2024-04-22T06:31:07.025420+0000","last_unstale":"2024-04-22T06:31:12.559643+0000","last_undegraded":"2024-04-22T06:31:12.559643+0000","last_fullsized":"2024-04-22T06:31:12.559643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:39:40.107621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560547+0000","last_change":"2024-04-22T06:31:06.005805+0000","last_active":"2024-04-22T06:31:12.560547+0000","last_peered":"2024-04-22T06:31:12.560547+0000","last_clean":"2024-04-22T06:31:12.560547+0000","last_became_active":"2024-04-22T06:31:06.005479+0000","last_became_peered":"2024-04-22T06:31:06.005479+0000","last_unstale":"2024-04-22T06:31:12.560547+0000","last_undegraded":"2024-04-22T06:31:12.560547+0000","last_fullsized":"2024-04-22T06:31:12.560547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:59:37.140612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"24'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558740+0000","last_change":"2024-04-22T06:31:05.998354+0000","last_active":"2024-04-22T06:31:12.558740+0000","last_peered":"2024-04-22T06:31:12.558740+0000","last_clean":"2024-04-22T06:31:12.558740+0000","last_became_active":"2024-04-22T06:31:05.998071+0000","last_became_peered":"2024-04-22T06:31:05.998071+0000","last_unstale":"2024-04-22T06:31:12.558740+0000","last_undegraded":"2024-04-22T06:31:12.558740+0000","last_fullsized":"2024-04-22T06:31:12.558740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:12:13.343435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561078+0000","last_change":"2024-04-22T06:31:07.024109+0000","last_active":"2024-04-22T06:31:12.561078+0000","last_peered":"2024-04-22T06:31:12.561078+0000","last_clean":"2024-04-22T06:31:12.561078+0000","last_became_active":"2024-04-22T06:31:07.023346+0000","last_became_peered":"2024-04-22T06:31:07.023346+0000","last_unstale":"2024-04-22T06:31:12.561078+0000","last_undegraded":"2024-04-22T06:31:12.561078+0000","last_fullsized":"2024-04-22T06:31:12.561078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:02:42.662312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559888+0000","last_change":"2024-04-22T06:31:11.047132+0000","last_active":"2024-04-22T06:31:12.559888+0000","last_peered":"2024-04-22T06:31:12.559888+0000","last_clean":"2024-04-22T06:31:12.559888+0000","last_became_active":"2024-04-22T06:31:11.046671+0000","last_became_peered":"2024-04-22T06:31:11.046671+0000","last_unstale":"2024-04-22T06:31:12.559888+0000","last_undegraded":"2024-04-22T06:31:12.559888+0000","last_fullsized":"2024-04-22T06:31:12.559888+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:41:57.706735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"25'5","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:37.094390+0000","last_change":"2024-04-22T06:31:06.004971+0000","last_active":"2024-04-22T06:31:37.094390+0000","last_peered":"2024-04-22T06:31:37.094390+0000","last_clean":"2024-04-22T06:31:37.094390+0000","last_became_active":"2024-04-22T06:31:06.004473+0000","last_became_peered":"2024-04-22T06:31:06.004473+0000","last_unstale":"2024-04-22T06:31:37.094390+0000","last_undegraded":"2024-04-22T06:31:37.094390+0000","last_fullsized":"2024-04-22T06:31:37.094390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:47:11.735311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.032950+0000","last_change":"2024-04-22T06:31:07.018314+0000","last_active":"2024-04-22T06:31:12.032950+0000","last_peered":"2024-04-22T06:31:12.032950+0000","last_clean":"2024-04-22T06:31:12.032950+0000","last_became_active":"2024-04-22T06:31:07.018033+0000","last_became_peered":"2024-04-22T06:31:07.018033+0000","last_unstale":"2024-04-22T06:31:12.032950+0000","last_undegraded":"2024-04-22T06:31:12.032950+0000","last_fullsized":"2024-04-22T06:31:12.032950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:06:06.099494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559988+0000","last_change":"2024-04-22T06:31:11.055874+0000","last_active":"2024-04-22T06:31:12.559988+0000","last_peered":"2024-04-22T06:31:12.559988+0000","last_clean":"2024-04-22T06:31:12.559988+0000","last_became_active":"2024-04-22T06:31:11.055371+0000","last_became_peered":"2024-04-22T06:31:11.055371+0000","last_unstale":"2024-04-22T06:31:12.559988+0000","last_undegraded":"2024-04-22T06:31:12.559988+0000","last_fullsized":"2024-04-22T06:31:12.559988+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:06:03.909337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558461+0000","last_change":"2024-04-22T06:31:07.024780+0000","last_active":"2024-04-22T06:31:12.558461+0000","last_peered":"2024-04-22T06:31:12.558461+0000","last_clean":"2024-04-22T06:31:12.558461+0000","last_became_active":"2024-04-22T06:31:07.024501+0000","last_became_peered":"2024-04-22T06:31:07.024501+0000","last_unstale":"2024-04-22T06:31:12.558461+0000","last_undegraded":"2024-04-22T06:31:12.558461+0000","last_fullsized":"2024-04-22T06:31:12.558461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:12:45.981232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558747+0000","last_change":"2024-04-22T06:31:07.014546+0000","last_active":"2024-04-22T06:31:12.558747+0000","last_peered":"2024-04-22T06:31:12.558747+0000","last_clean":"2024-04-22T06:31:12.558747+0000","last_became_active":"2024-04-22T06:31:07.014196+0000","last_became_peered":"2024-04-22T06:31:07.014196+0000","last_unstale":"2024-04-22T06:31:12.558747+0000","last_undegraded":"2024-04-22T06:31:12.558747+0000","last_fullsized":"2024-04-22T06:31:12.558747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:57:33.136875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031490+0000","last_change":"2024-04-22T06:31:07.018563+0000","last_active":"2024-04-22T06:31:12.031490+0000","last_peered":"2024-04-22T06:31:12.031490+0000","last_clean":"2024-04-22T06:31:12.031490+0000","last_became_active":"2024-04-22T06:31:07.018289+0000","last_became_peered":"2024-04-22T06:31:07.018289+0000","last_unstale":"2024-04-22T06:31:12.031490+0000","last_undegraded":"2024-04-22T06:31:12.031490+0000","last_fullsized":"2024-04-22T06:31:12.031490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:30:11.892544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560983+0000","last_change":"2024-04-22T06:31:07.022901+0000","last_active":"2024-04-22T06:31:12.560983+0000","last_peered":"2024-04-22T06:31:12.560983+0000","last_clean":"2024-04-22T06:31:12.560983+0000","last_became_active":"2024-04-22T06:31:07.022420+0000","last_became_peered":"2024-04-22T06:31:07.022420+0000","last_unstale":"2024-04-22T06:31:12.560983+0000","last_undegraded":"2024-04-22T06:31:12.560983+0000","last_fullsized":"2024-04-22T06:31:12.560983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:24:18.929896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559253+0000","last_change":"2024-04-22T06:31:07.022875+0000","last_active":"2024-04-22T06:31:12.559253+0000","last_peered":"2024-04-22T06:31:12.559253+0000","last_clean":"2024-04-22T06:31:12.559253+0000","last_became_active":"2024-04-22T06:31:07.022545+0000","last_became_peered":"2024-04-22T06:31:07.022545+0000","last_unstale":"2024-04-22T06:31:12.559253+0000","last_undegraded":"2024-04-22T06:31:12.559253+0000","last_fullsized":"2024-04-22T06:31:12.559253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:02:30.464508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560354+0000","last_change":"2024-04-22T06:31:07.026367+0000","last_active":"2024-04-22T06:31:12.560354+0000","last_peered":"2024-04-22T06:31:12.560354+0000","last_clean":"2024-04-22T06:31:12.560354+0000","last_became_active":"2024-04-22T06:31:07.026085+0000","last_became_peered":"2024-04-22T06:31:07.026085+0000","last_unstale":"2024-04-22T06:31:12.560354+0000","last_undegraded":"2024-04-22T06:31:12.560354+0000","last_fullsized":"2024-04-22T06:31:12.560354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:02:55.368591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559243+0000","last_change":"2024-04-22T06:31:07.022989+0000","last_active":"2024-04-22T06:31:12.559243+0000","last_peered":"2024-04-22T06:31:12.559243+0000","last_clean":"2024-04-22T06:31:12.559243+0000","last_became_active":"2024-04-22T06:31:07.022688+0000","last_became_peered":"2024-04-22T06:31:07.022688+0000","last_unstale":"2024-04-22T06:31:12.559243+0000","last_undegraded":"2024-04-22T06:31:12.559243+0000","last_fullsized":"2024-04-22T06:31:12.559243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:24:10.910731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559590+0000","last_change":"2024-04-22T06:31:07.026315+0000","last_active":"2024-04-22T06:31:12.559590+0000","last_peered":"2024-04-22T06:31:12.559590+0000","last_clean":"2024-04-22T06:31:12.559590+0000","last_became_active":"2024-04-22T06:31:07.025951+0000","last_became_peered":"2024-04-22T06:31:07.025951+0000","last_unstale":"2024-04-22T06:31:12.559590+0000","last_undegraded":"2024-04-22T06:31:12.559590+0000","last_fullsized":"2024-04-22T06:31:12.559590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:54:12.160058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030585+0000","last_change":"2024-04-22T06:31:07.022748+0000","last_active":"2024-04-22T06:31:12.030585+0000","last_peered":"2024-04-22T06:31:12.030585+0000","last_clean":"2024-04-22T06:31:12.030585+0000","last_became_active":"2024-04-22T06:31:07.022481+0000","last_became_peered":"2024-04-22T06:31:07.022481+0000","last_unstale":"2024-04-22T06:31:12.030585+0000","last_undegraded":"2024-04-22T06:31:12.030585+0000","last_fullsized":"2024-04-22T06:31:12.030585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:56:32.631637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560456+0000","last_change":"2024-04-22T06:31:07.021939+0000","last_active":"2024-04-22T06:31:12.560456+0000","last_peered":"2024-04-22T06:31:12.560456+0000","last_clean":"2024-04-22T06:31:12.560456+0000","last_became_active":"2024-04-22T06:31:07.021638+0000","last_became_peered":"2024-04-22T06:31:07.021638+0000","last_unstale":"2024-04-22T06:31:12.560456+0000","last_undegraded":"2024-04-22T06:31:12.560456+0000","last_fullsized":"2024-04-22T06:31:12.560456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:05:27.106092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031390+0000","last_change":"2024-04-22T06:31:07.015844+0000","last_active":"2024-04-22T06:31:12.031390+0000","last_peered":"2024-04-22T06:31:12.031390+0000","last_clean":"2024-04-22T06:31:12.031390+0000","last_became_active":"2024-04-22T06:31:07.015520+0000","last_became_peered":"2024-04-22T06:31:07.015520+0000","last_unstale":"2024-04-22T06:31:12.031390+0000","last_undegraded":"2024-04-22T06:31:12.031390+0000","last_fullsized":"2024-04-22T06:31:12.031390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:24:04.676892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560128+0000","last_change":"2024-04-22T06:31:07.023697+0000","last_active":"2024-04-22T06:31:12.560128+0000","last_peered":"2024-04-22T06:31:12.560128+0000","last_clean":"2024-04-22T06:31:12.560128+0000","last_became_active":"2024-04-22T06:31:07.023283+0000","last_became_peered":"2024-04-22T06:31:07.023283+0000","last_unstale":"2024-04-22T06:31:12.560128+0000","last_undegraded":"2024-04-22T06:31:12.560128+0000","last_fullsized":"2024-04-22T06:31:12.560128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:43:01.338104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558868+0000","last_change":"2024-04-22T06:31:07.023953+0000","last_active":"2024-04-22T06:31:12.558868+0000","last_peered":"2024-04-22T06:31:12.558868+0000","last_clean":"2024-04-22T06:31:12.558868+0000","last_became_active":"2024-04-22T06:31:07.023704+0000","last_became_peered":"2024-04-22T06:31:07.023704+0000","last_unstale":"2024-04-22T06:31:12.558868+0000","last_undegraded":"2024-04-22T06:31:12.558868+0000","last_fullsized":"2024-04-22T06:31:12.558868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:59:32.219492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031619+0000","last_change":"2024-04-22T06:31:07.027418+0000","last_active":"2024-04-22T06:31:12.031619+0000","last_peered":"2024-04-22T06:31:12.031619+0000","last_clean":"2024-04-22T06:31:12.031619+0000","last_became_active":"2024-04-22T06:31:07.027113+0000","last_became_peered":"2024-04-22T06:31:07.027113+0000","last_unstale":"2024-04-22T06:31:12.031619+0000","last_undegraded":"2024-04-22T06:31:12.031619+0000","last_fullsized":"2024-04-22T06:31:12.031619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:08:42.204674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559261+0000","last_change":"2024-04-22T06:31:07.015520+0000","last_active":"2024-04-22T06:31:12.559261+0000","last_peered":"2024-04-22T06:31:12.559261+0000","last_clean":"2024-04-22T06:31:12.559261+0000","last_became_active":"2024-04-22T06:31:07.015199+0000","last_became_peered":"2024-04-22T06:31:07.015199+0000","last_unstale":"2024-04-22T06:31:12.559261+0000","last_undegraded":"2024-04-22T06:31:12.559261+0000","last_fullsized":"2024-04-22T06:31:12.559261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:48:40.688931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031281+0000","last_change":"2024-04-22T06:31:07.016457+0000","last_active":"2024-04-22T06:31:12.031281+0000","last_peered":"2024-04-22T06:31:12.031281+0000","last_clean":"2024-04-22T06:31:12.031281+0000","last_became_active":"2024-04-22T06:31:07.016082+0000","last_became_peered":"2024-04-22T06:31:07.016082+0000","last_unstale":"2024-04-22T06:31:12.031281+0000","last_undegraded":"2024-04-22T06:31:12.031281+0000","last_fullsized":"2024-04-22T06:31:12.031281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:12:28.517065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030497+0000","last_change":"2024-04-22T06:31:07.019955+0000","last_active":"2024-04-22T06:31:12.030497+0000","last_peered":"2024-04-22T06:31:12.030497+0000","last_clean":"2024-04-22T06:31:12.030497+0000","last_became_active":"2024-04-22T06:31:07.019174+0000","last_became_peered":"2024-04-22T06:31:07.019174+0000","last_unstale":"2024-04-22T06:31:12.030497+0000","last_undegraded":"2024-04-22T06:31:12.030497+0000","last_fullsized":"2024-04-22T06:31:12.030497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:09:07.106161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560474+0000","last_change":"2024-04-22T06:31:07.019341+0000","last_active":"2024-04-22T06:31:12.560474+0000","last_peered":"2024-04-22T06:31:12.560474+0000","last_clean":"2024-04-22T06:31:12.560474+0000","last_became_active":"2024-04-22T06:31:07.019048+0000","last_became_peered":"2024-04-22T06:31:07.019048+0000","last_unstale":"2024-04-22T06:31:12.560474+0000","last_undegraded":"2024-04-22T06:31:12.560474+0000","last_fullsized":"2024-04-22T06:31:12.560474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:04:19.362686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559337+0000","last_change":"2024-04-22T06:31:07.019433+0000","last_active":"2024-04-22T06:31:12.559337+0000","last_peered":"2024-04-22T06:31:12.559337+0000","last_clean":"2024-04-22T06:31:12.559337+0000","last_became_active":"2024-04-22T06:31:07.019133+0000","last_became_peered":"2024-04-22T06:31:07.019133+0000","last_unstale":"2024-04-22T06:31:12.559337+0000","last_undegraded":"2024-04-22T06:31:12.559337+0000","last_fullsized":"2024-04-22T06:31:12.559337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:01:54.466486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561087+0000","last_change":"2024-04-22T06:31:07.023014+0000","last_active":"2024-04-22T06:31:12.561087+0000","last_peered":"2024-04-22T06:31:12.561087+0000","last_clean":"2024-04-22T06:31:12.561087+0000","last_became_active":"2024-04-22T06:31:07.022716+0000","last_became_peered":"2024-04-22T06:31:07.022716+0000","last_unstale":"2024-04-22T06:31:12.561087+0000","last_undegraded":"2024-04-22T06:31:12.561087+0000","last_fullsized":"2024-04-22T06:31:12.561087+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:11:47.309706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.033363+0000","last_change":"2024-04-22T06:31:07.020446+0000","last_active":"2024-04-22T06:31:12.033363+0000","last_peered":"2024-04-22T06:31:12.033363+0000","last_clean":"2024-04-22T06:31:12.033363+0000","last_became_active":"2024-04-22T06:31:07.018995+0000","last_became_peered":"2024-04-22T06:31:07.018995+0000","last_unstale":"2024-04-22T06:31:12.033363+0000","last_undegraded":"2024-04-22T06:31:12.033363+0000","last_fullsized":"2024-04-22T06:31:12.033363+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:56:45.031262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558848+0000","last_change":"2024-04-22T06:31:07.014927+0000","last_active":"2024-04-22T06:31:12.558848+0000","last_peered":"2024-04-22T06:31:12.558848+0000","last_clean":"2024-04-22T06:31:12.558848+0000","last_became_active":"2024-04-22T06:31:07.014652+0000","last_became_peered":"2024-04-22T06:31:07.014652+0000","last_unstale":"2024-04-22T06:31:12.558848+0000","last_undegraded":"2024-04-22T06:31:12.558848+0000","last_fullsized":"2024-04-22T06:31:12.558848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:51:33.506667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559161+0000","last_change":"2024-04-22T06:31:07.014209+0000","last_active":"2024-04-22T06:31:12.559161+0000","last_peered":"2024-04-22T06:31:12.559161+0000","last_clean":"2024-04-22T06:31:12.559161+0000","last_became_active":"2024-04-22T06:31:07.013946+0000","last_became_peered":"2024-04-22T06:31:07.013946+0000","last_unstale":"2024-04-22T06:31:12.559161+0000","last_undegraded":"2024-04-22T06:31:12.559161+0000","last_fullsized":"2024-04-22T06:31:12.559161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:40:31.669158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031004+0000","last_change":"2024-04-22T06:31:07.008764+0000","last_active":"2024-04-22T06:31:12.031004+0000","last_peered":"2024-04-22T06:31:12.031004+0000","last_clean":"2024-04-22T06:31:12.031004+0000","last_became_active":"2024-04-22T06:31:07.007810+0000","last_became_peered":"2024-04-22T06:31:07.007810+0000","last_unstale":"2024-04-22T06:31:12.031004+0000","last_undegraded":"2024-04-22T06:31:12.031004+0000","last_fullsized":"2024-04-22T06:31:12.031004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:49:54.894818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559408+0000","last_change":"2024-04-22T06:31:07.022141+0000","last_active":"2024-04-22T06:31:12.559408+0000","last_peered":"2024-04-22T06:31:12.559408+0000","last_clean":"2024-04-22T06:31:12.559408+0000","last_became_active":"2024-04-22T06:31:07.021845+0000","last_became_peered":"2024-04-22T06:31:07.021845+0000","last_unstale":"2024-04-22T06:31:12.559408+0000","last_undegraded":"2024-04-22T06:31:12.559408+0000","last_fullsized":"2024-04-22T06:31:12.559408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:26:28.598104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.029984+0000","last_change":"2024-04-22T06:31:07.012188+0000","last_active":"2024-04-22T06:31:12.029984+0000","last_peered":"2024-04-22T06:31:12.029984+0000","last_clean":"2024-04-22T06:31:12.029984+0000","last_became_active":"2024-04-22T06:31:07.011909+0000","last_became_peered":"2024-04-22T06:31:07.011909+0000","last_unstale":"2024-04-22T06:31:12.029984+0000","last_undegraded":"2024-04-22T06:31:12.029984+0000","last_fullsized":"2024-04-22T06:31:12.029984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:25:13.089928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031293+0000","last_change":"2024-04-22T06:31:07.020033+0000","last_active":"2024-04-22T06:31:12.031293+0000","last_peered":"2024-04-22T06:31:12.031293+0000","last_clean":"2024-04-22T06:31:12.031293+0000","last_became_active":"2024-04-22T06:31:07.019689+0000","last_became_peered":"2024-04-22T06:31:07.019689+0000","last_unstale":"2024-04-22T06:31:12.031293+0000","last_undegraded":"2024-04-22T06:31:12.031293+0000","last_fullsized":"2024-04-22T06:31:12.031293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:09:07.209203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030666+0000","last_change":"2024-04-22T06:31:07.027287+0000","last_active":"2024-04-22T06:31:12.030666+0000","last_peered":"2024-04-22T06:31:12.030666+0000","last_clean":"2024-04-22T06:31:12.030666+0000","last_became_active":"2024-04-22T06:31:07.027015+0000","last_became_peered":"2024-04-22T06:31:07.027015+0000","last_unstale":"2024-04-22T06:31:12.030666+0000","last_undegraded":"2024-04-22T06:31:12.030666+0000","last_fullsized":"2024-04-22T06:31:12.030666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:42:49.322413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031959+0000","last_change":"2024-04-22T06:31:07.026332+0000","last_active":"2024-04-22T06:31:12.031959+0000","last_peered":"2024-04-22T06:31:12.031959+0000","last_clean":"2024-04-22T06:31:12.031959+0000","last_became_active":"2024-04-22T06:31:07.026018+0000","last_became_peered":"2024-04-22T06:31:07.026018+0000","last_unstale":"2024-04-22T06:31:12.031959+0000","last_undegraded":"2024-04-22T06:31:12.031959+0000","last_fullsized":"2024-04-22T06:31:12.031959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:27:08.560056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560402+0000","last_change":"2024-04-22T06:31:07.020497+0000","last_active":"2024-04-22T06:31:12.560402+0000","last_peered":"2024-04-22T06:31:12.560402+0000","last_clean":"2024-04-22T06:31:12.560402+0000","last_became_active":"2024-04-22T06:31:07.020194+0000","last_became_peered":"2024-04-22T06:31:07.020194+0000","last_unstale":"2024-04-22T06:31:12.560402+0000","last_undegraded":"2024-04-22T06:31:12.560402+0000","last_fullsized":"2024-04-22T06:31:12.560402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:56:59.765295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560664+0000","last_change":"2024-04-22T06:31:07.018396+0000","last_active":"2024-04-22T06:31:12.560664+0000","last_peered":"2024-04-22T06:31:12.560664+0000","last_clean":"2024-04-22T06:31:12.560664+0000","last_became_active":"2024-04-22T06:31:07.018073+0000","last_became_peered":"2024-04-22T06:31:07.018073+0000","last_unstale":"2024-04-22T06:31:12.560664+0000","last_undegraded":"2024-04-22T06:31:12.560664+0000","last_fullsized":"2024-04-22T06:31:12.560664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:37:06.501499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030917+0000","last_change":"2024-04-22T06:31:07.014439+0000","last_active":"2024-04-22T06:31:12.030917+0000","last_peered":"2024-04-22T06:31:12.030917+0000","last_clean":"2024-04-22T06:31:12.030917+0000","last_became_active":"2024-04-22T06:31:07.014124+0000","last_became_peered":"2024-04-22T06:31:07.014124+0000","last_unstale":"2024-04-22T06:31:12.030917+0000","last_undegraded":"2024-04-22T06:31:12.030917+0000","last_fullsized":"2024-04-22T06:31:12.030917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:58:46.075068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6046,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":12092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":880640,"data_stored":696611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":282624,"data_stored":108859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":278528,"data_stored":105689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":278528,"data_stored":105053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":286720,"data_stored":109391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574860,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":105759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":876544,"data_stored":695955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T06:31:40.770 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T06:31:40.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.908+0000 7f84a4398640 1 Processor -- start 2024-04-22T06:31:40.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.908+0000 7f84a4398640 1 -- start start 2024-04-22T06:31:40.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.909+0000 7f84a4398640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f849c0c5ee0 0x7f849c1598e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:40.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.909+0000 7f84a4398640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f849c0c62c0 0x7f849c1528e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:40.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.909+0000 7f84a4398640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f849c159e20 con 0x7f849c0c9970 2024-04-22T06:31:40.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.909+0000 7f84a4398640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f849c15d340 con 0x7f849c0c5ee0 2024-04-22T06:31:40.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.909+0000 7f84a4398640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f849c0c9e40 con 0x7f849c0c62c0 2024-04-22T06:31:40.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.909+0000 7f84a2134640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f849c0c5ee0 0x7f849c1598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:40.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.909+0000 7f84a1933640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f849c0c9970 0x7f849c071300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:49536/0 (socket says 172.21.15.44:49536) 2024-04-22T06:31:40.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.909+0000 7f84a1933640 1 -- 172.21.15.44:0/3441368632 learned_addr learned my addr 172.21.15.44:0/3441368632 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:40.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.909+0000 7f84a2935640 1 --2- 172.21.15.44:0/3441368632 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f849c0c62c0 0x7f849c1528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:40.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.909+0000 7f84a1132640 1 -- 172.21.15.44:0/3441368632 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3721217812 0 0) 0x7f848c001110 con 0x7f849c0c9970 2024-04-22T06:31:40.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.909+0000 7f84a1132640 1 -- 172.21.15.44:0/3441368632 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8488003520 con 0x7f849c0c9970 2024-04-22T06:31:40.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.909+0000 7f84a2134640 1 -- 172.21.15.44:0/3441368632 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f849c0c62c0 msgr2=0x7f849c1528e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:40.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.909+0000 7f84a2134640 1 --2- 172.21.15.44:0/3441368632 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f849c0c62c0 0x7f849c1528e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:40.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.909+0000 7f84a2134640 1 -- 172.21.15.44:0/3441368632 >> v1:172.21.15.44:6789/0 conn(0x7f849c0c9970 legacy=0x7f849c071300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:40.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.909+0000 7f84a2134640 1 -- 172.21.15.44:0/3441368632 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f849c15c450 con 0x7f849c0c5ee0 2024-04-22T06:31:40.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.909+0000 7f84a2935640 1 --2- 172.21.15.44:0/3441368632 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f849c0c62c0 0x7f849c1528e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:31:40.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.910+0000 7f84a2134640 1 --2- 172.21.15.44:0/3441368632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f849c0c5ee0 0x7f849c1598e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f84980097a0 tx=0x7f849802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a35779a7fa8cd74d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:40.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.910+0000 7f84a1132640 1 -- 172.21.15.44:0/3441368632 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8498034070 con 0x7f849c0c5ee0 2024-04-22T06:31:40.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.910+0000 7f84a1132640 1 -- 172.21.15.44:0/3441368632 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8498030610 con 0x7f849c0c5ee0 2024-04-22T06:31:40.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.910+0000 7f84a1132640 1 -- 172.21.15.44:0/3441368632 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84980309e0 con 0x7f849c0c5ee0 2024-04-22T06:31:40.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.911+0000 7f84a4398640 1 -- 172.21.15.44:0/3441368632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f849c0c5ee0 msgr2=0x7f849c1598e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:40.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.911+0000 7f84a4398640 1 --2- 172.21.15.44:0/3441368632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f849c0c5ee0 0x7f849c1598e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f84980097a0 tx=0x7f849802b960 comp rx=0 tx=0).stop 2024-04-22T06:31:40.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.911+0000 7f84a4398640 1 -- 172.21.15.44:0/3441368632 shutdown_connections 2024-04-22T06:31:40.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.911+0000 7f84a4398640 1 --2- 172.21.15.44:0/3441368632 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f849c0c62c0 0x7f849c1528e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:40.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.911+0000 7f84a4398640 1 --2- 172.21.15.44:0/3441368632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f849c0c5ee0 0x7f849c1598e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:40.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.911+0000 7f84a4398640 1 -- 172.21.15.44:0/3441368632 >> 172.21.15.44:0/3441368632 conn(0x7f849c074cb0 msgr2=0x7f849c0750c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:40.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.911+0000 7f84a4398640 1 -- 172.21.15.44:0/3441368632 shutdown_connections 2024-04-22T06:31:40.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.911+0000 7f84a4398640 1 -- 172.21.15.44:0/3441368632 wait complete. 2024-04-22T06:31:40.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.912+0000 7f84a4398640 1 Processor -- start 2024-04-22T06:31:40.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.912+0000 7f84a4398640 1 -- start start 2024-04-22T06:31:40.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.912+0000 7f84a4398640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f849c0c5ee0 0x7f849c1f2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:40.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.912+0000 7f84a4398640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f849c0c62c0 0x7f849c14f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:40.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.912+0000 7f84a4398640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f849c0c9970 0x7f849c0cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:40.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.912+0000 7f84a4398640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f849c15ccf0 con 0x7f849c0c9970 2024-04-22T06:31:40.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.912+0000 7f84a4398640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f849c15ca10 con 0x7f849c0c62c0 2024-04-22T06:31:40.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.912+0000 7f84a4398640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f849c15cfd0 con 0x7f849c0c5ee0 2024-04-22T06:31:40.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.913+0000 7f84a2935640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f849c0c9970 0x7f849c0cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:40.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.913+0000 7f84a2935640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f849c0c9970 0x7f849c0cbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53932/0 (socket says 172.21.15.44:53932) 2024-04-22T06:31:40.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.913+0000 7f84a2935640 1 -- 172.21.15.44:0/4127466879 learned_addr learned my addr 172.21.15.44:0/4127466879 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:40.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.913+0000 7f84a1933640 1 --2- 172.21.15.44:0/4127466879 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f849c0c62c0 0x7f849c14f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:40.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.913+0000 7f84a2134640 1 --2- 172.21.15.44:0/4127466879 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f849c0c5ee0 0x7f849c1f2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:40.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.913+0000 7f84a2935640 1 -- 172.21.15.44:0/4127466879 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f849c0c5ee0 msgr2=0x7f849c1f2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:40.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.913+0000 7f84a2935640 1 --2- 172.21.15.44:0/4127466879 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f849c0c5ee0 0x7f849c1f2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:40.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.913+0000 7f84a2935640 1 -- 172.21.15.44:0/4127466879 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f849c0c62c0 msgr2=0x7f849c14f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:40.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.913+0000 7f84a2935640 1 --2- 172.21.15.44:0/4127466879 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f849c0c62c0 0x7f849c14f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:40.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.913+0000 7f84a2935640 1 -- 172.21.15.44:0/4127466879 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f849c1f2c50 con 0x7f849c0c9970 2024-04-22T06:31:40.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.913+0000 7f84a2935640 1 --2- 172.21.15.44:0/4127466879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f849c0c9970 0x7f849c0cbbd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8494009f00 tx=0x7f849400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:40.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.913+0000 7f8492ffd640 1 -- 172.21.15.44:0/4127466879 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8494018030 con 0x7f849c0c9970 2024-04-22T06:31:40.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.913+0000 7f8492ffd640 1 -- 172.21.15.44:0/4127466879 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f849400ce50 con 0x7f849c0c9970 2024-04-22T06:31:40.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.913+0000 7f84a4398640 1 -- 172.21.15.44:0/4127466879 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f849c1582f0 con 0x7f849c0c9970 2024-04-22T06:31:40.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.913+0000 7f8492ffd640 1 -- 172.21.15.44:0/4127466879 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8494003bb0 con 0x7f849c0c9970 2024-04-22T06:31:40.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.913+0000 7f84a4398640 1 -- 172.21.15.44:0/4127466879 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f849c157ba0 con 0x7f849c0c9970 2024-04-22T06:31:40.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.914+0000 7f84a4398640 1 -- 172.21.15.44:0/4127466879 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f849c068cc0 con 0x7f849c0c9970 2024-04-22T06:31:40.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.915+0000 7f8492ffd640 1 -- 172.21.15.44:0/4127466879 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8494005d10 con 0x7f849c0c9970 2024-04-22T06:31:40.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.915+0000 7f8492ffd640 1 --2- 172.21.15.44:0/4127466879 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f848006c920 0x7f848006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:40.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.915+0000 7f8492ffd640 1 -- 172.21.15.44:0/4127466879 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f849408c710 con 0x7f849c0c9970 2024-04-22T06:31:40.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.915+0000 7f84a2134640 1 --2- 172.21.15.44:0/4127466879 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f848006c920 0x7f848006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:40.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.916+0000 7f84a2134640 1 --2- 172.21.15.44:0/4127466879 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f848006c920 0x7f848006ede0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8498036000 tx=0x7f849803b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:40.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:40.919+0000 7f8492ffd640 1 -- 172.21.15.44:0/4127466879 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8494022070 con 0x7f849c0c9970 2024-04-22T06:31:41.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.100+0000 7f84a4398640 1 -- 172.21.15.44:0/4127466879 --> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f849c1519e0 con 0x7f848006c920 2024-04-22T06:31:41.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.115+0000 7f8492ffd640 1 -- 172.21.15.44:0/4127466879 <== mgr.4111 v2:172.21.15.70:6832/2570466131 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332672 (secure 0 0 0) 0x7f8498099390 con 0x7f848006c920 2024-04-22T06:31:41.118 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:31:41.119 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-04-22T06:31:41.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.119+0000 7f84a4398640 1 -- 172.21.15.44:0/4127466879 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f848006c920 msgr2=0x7f848006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:41.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.119+0000 7f84a4398640 1 --2- 172.21.15.44:0/4127466879 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f848006c920 0x7f848006ede0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8498036000 tx=0x7f849803b040 comp rx=0 tx=0).stop 2024-04-22T06:31:41.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.119+0000 7f84a4398640 1 -- 172.21.15.44:0/4127466879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f849c0c9970 msgr2=0x7f849c0cbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:41.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.119+0000 7f84a4398640 1 --2- 172.21.15.44:0/4127466879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f849c0c9970 0x7f849c0cbbd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8494009f00 tx=0x7f849400c490 comp rx=0 tx=0).stop 2024-04-22T06:31:41.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.119+0000 7f84a4398640 1 -- 172.21.15.44:0/4127466879 shutdown_connections 2024-04-22T06:31:41.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.119+0000 7f84a4398640 1 --2- 172.21.15.44:0/4127466879 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f848006c920 0x7f848006ede0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:41.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.119+0000 7f84a4398640 1 --2- 172.21.15.44:0/4127466879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f849c0c9970 0x7f849c0cbbd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:41.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.119+0000 7f84a4398640 1 --2- 172.21.15.44:0/4127466879 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f849c0c62c0 0x7f849c14f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:41.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.119+0000 7f84a4398640 1 --2- 172.21.15.44:0/4127466879 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f849c0c5ee0 0x7f849c1f2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:41.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.119+0000 7f84a4398640 1 -- 172.21.15.44:0/4127466879 >> 172.21.15.44:0/4127466879 conn(0x7f849c074cb0 msgr2=0x7f849c14cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:41.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.119+0000 7f84a4398640 1 -- 172.21.15.44:0/4127466879 shutdown_connections 2024-04-22T06:31:41.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.119+0000 7f84a4398640 1 -- 172.21.15.44:0/4127466879 wait complete. 2024-04-22T06:31:41.128 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2024-04-22T06:31:40.553336+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596414,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":246,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":193,"ondisk_log_size":193,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236120,"kb_used_data":3352,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738600,"statfs":{"total":773094113280,"available":772852326400,"internally_reserved":0,"allocated":3432448,"data_stored":2032244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1406,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002774"},"pg_stats":[{"pgid":"2.1f","version":"24'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560936+0000","last_change":"2024-04-22T06:31:06.002169+0000","last_active":"2024-04-22T06:31:12.560936+0000","last_peered":"2024-04-22T06:31:12.560936+0000","last_clean":"2024-04-22T06:31:12.560936+0000","last_became_active":"2024-04-22T06:31:06.001899+0000","last_became_peered":"2024-04-22T06:31:06.001899+0000","last_unstale":"2024-04-22T06:31:12.560936+0000","last_undegraded":"2024-04-22T06:31:12.560936+0000","last_fullsized":"2024-04-22T06:31:12.560936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:11:48.056404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559298+0000","last_change":"2024-04-22T06:31:07.025982+0000","last_active":"2024-04-22T06:31:12.559298+0000","last_peered":"2024-04-22T06:31:12.559298+0000","last_clean":"2024-04-22T06:31:12.559298+0000","last_became_active":"2024-04-22T06:31:07.025650+0000","last_became_peered":"2024-04-22T06:31:07.025650+0000","last_unstale":"2024-04-22T06:31:12.559298+0000","last_undegraded":"2024-04-22T06:31:12.559298+0000","last_fullsized":"2024-04-22T06:31:12.559298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:58:58.029898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560389+0000","last_change":"2024-04-22T06:31:11.048083+0000","last_active":"2024-04-22T06:31:12.560389+0000","last_peered":"2024-04-22T06:31:12.560389+0000","last_clean":"2024-04-22T06:31:12.560389+0000","last_became_active":"2024-04-22T06:31:11.047682+0000","last_became_peered":"2024-04-22T06:31:11.047682+0000","last_unstale":"2024-04-22T06:31:12.560389+0000","last_undegraded":"2024-04-22T06:31:12.560389+0000","last_fullsized":"2024-04-22T06:31:12.560389+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:34:30.158935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"24'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559543+0000","last_change":"2024-04-22T06:31:06.007484+0000","last_active":"2024-04-22T06:31:12.559543+0000","last_peered":"2024-04-22T06:31:12.559543+0000","last_clean":"2024-04-22T06:31:12.559543+0000","last_became_active":"2024-04-22T06:31:06.007204+0000","last_became_peered":"2024-04-22T06:31:06.007204+0000","last_unstale":"2024-04-22T06:31:12.559543+0000","last_undegraded":"2024-04-22T06:31:12.559543+0000","last_fullsized":"2024-04-22T06:31:12.559543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:39:18.905813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560596+0000","last_change":"2024-04-22T06:31:07.014658+0000","last_active":"2024-04-22T06:31:12.560596+0000","last_peered":"2024-04-22T06:31:12.560596+0000","last_clean":"2024-04-22T06:31:12.560596+0000","last_became_active":"2024-04-22T06:31:07.014283+0000","last_became_peered":"2024-04-22T06:31:07.014283+0000","last_unstale":"2024-04-22T06:31:12.560596+0000","last_undegraded":"2024-04-22T06:31:12.560596+0000","last_fullsized":"2024-04-22T06:31:12.560596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:43:01.159439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.032692+0000","last_change":"2024-04-22T06:31:11.045114+0000","last_active":"2024-04-22T06:31:12.032692+0000","last_peered":"2024-04-22T06:31:12.032692+0000","last_clean":"2024-04-22T06:31:12.032692+0000","last_became_active":"2024-04-22T06:31:11.044728+0000","last_became_peered":"2024-04-22T06:31:11.044728+0000","last_unstale":"2024-04-22T06:31:12.032692+0000","last_undegraded":"2024-04-22T06:31:12.032692+0000","last_fullsized":"2024-04-22T06:31:12.032692+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:48:19.215184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"24'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031494+0000","last_change":"2024-04-22T06:31:05.995468+0000","last_active":"2024-04-22T06:31:12.031494+0000","last_peered":"2024-04-22T06:31:12.031494+0000","last_clean":"2024-04-22T06:31:12.031494+0000","last_became_active":"2024-04-22T06:31:05.995159+0000","last_became_peered":"2024-04-22T06:31:05.995159+0000","last_unstale":"2024-04-22T06:31:12.031494+0000","last_undegraded":"2024-04-22T06:31:12.031494+0000","last_fullsized":"2024-04-22T06:31:12.031494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:01:12.223718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031966+0000","last_change":"2024-04-22T06:31:07.020024+0000","last_active":"2024-04-22T06:31:12.031966+0000","last_peered":"2024-04-22T06:31:12.031966+0000","last_clean":"2024-04-22T06:31:12.031966+0000","last_became_active":"2024-04-22T06:31:07.019490+0000","last_became_peered":"2024-04-22T06:31:07.019490+0000","last_unstale":"2024-04-22T06:31:12.031966+0000","last_undegraded":"2024-04-22T06:31:12.031966+0000","last_fullsized":"2024-04-22T06:31:12.031966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:26:41.811978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.033365+0000","last_change":"2024-04-22T06:31:11.057556+0000","last_active":"2024-04-22T06:31:12.033365+0000","last_peered":"2024-04-22T06:31:12.033365+0000","last_clean":"2024-04-22T06:31:12.033365+0000","last_became_active":"2024-04-22T06:31:11.057174+0000","last_became_peered":"2024-04-22T06:31:11.057174+0000","last_unstale":"2024-04-22T06:31:12.033365+0000","last_undegraded":"2024-04-22T06:31:12.033365+0000","last_fullsized":"2024-04-22T06:31:12.033365+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:41:19.526548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030542+0000","last_change":"2024-04-22T06:31:05.996281+0000","last_active":"2024-04-22T06:31:12.030542+0000","last_peered":"2024-04-22T06:31:12.030542+0000","last_clean":"2024-04-22T06:31:12.030542+0000","last_became_active":"2024-04-22T06:31:05.995951+0000","last_became_peered":"2024-04-22T06:31:05.995951+0000","last_unstale":"2024-04-22T06:31:12.030542+0000","last_undegraded":"2024-04-22T06:31:12.030542+0000","last_fullsized":"2024-04-22T06:31:12.030542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:01:43.732947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.032029+0000","last_change":"2024-04-22T06:31:07.017755+0000","last_active":"2024-04-22T06:31:12.032029+0000","last_peered":"2024-04-22T06:31:12.032029+0000","last_clean":"2024-04-22T06:31:12.032029+0000","last_became_active":"2024-04-22T06:31:07.017420+0000","last_became_peered":"2024-04-22T06:31:07.017420+0000","last_unstale":"2024-04-22T06:31:12.032029+0000","last_undegraded":"2024-04-22T06:31:12.032029+0000","last_fullsized":"2024-04-22T06:31:12.032029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:30:57.913020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.032802+0000","last_change":"2024-04-22T06:31:11.049497+0000","last_active":"2024-04-22T06:31:12.032802+0000","last_peered":"2024-04-22T06:31:12.032802+0000","last_clean":"2024-04-22T06:31:12.032802+0000","last_became_active":"2024-04-22T06:31:11.049079+0000","last_became_peered":"2024-04-22T06:31:11.049079+0000","last_unstale":"2024-04-22T06:31:12.032802+0000","last_undegraded":"2024-04-22T06:31:12.032802+0000","last_fullsized":"2024-04-22T06:31:12.032802+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:29:02.299691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558561+0000","last_change":"2024-04-22T06:31:07.023043+0000","last_active":"2024-04-22T06:31:12.558561+0000","last_peered":"2024-04-22T06:31:12.558561+0000","last_clean":"2024-04-22T06:31:12.558561+0000","last_became_active":"2024-04-22T06:31:07.022747+0000","last_became_peered":"2024-04-22T06:31:07.022747+0000","last_unstale":"2024-04-22T06:31:12.558561+0000","last_undegraded":"2024-04-22T06:31:12.558561+0000","last_fullsized":"2024-04-22T06:31:12.558561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:52:33.626072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"25'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:37.088829+0000","last_change":"2024-04-22T06:31:05.997936+0000","last_active":"2024-04-22T06:31:37.088829+0000","last_peered":"2024-04-22T06:31:37.088829+0000","last_clean":"2024-04-22T06:31:37.088829+0000","last_became_active":"2024-04-22T06:31:05.997618+0000","last_became_peered":"2024-04-22T06:31:05.997618+0000","last_unstale":"2024-04-22T06:31:37.088829+0000","last_undegraded":"2024-04-22T06:31:37.088829+0000","last_fullsized":"2024-04-22T06:31:37.088829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:43:00.396691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.033550+0000","last_change":"2024-04-22T06:31:11.047725+0000","last_active":"2024-04-22T06:31:12.033550+0000","last_peered":"2024-04-22T06:31:12.033550+0000","last_clean":"2024-04-22T06:31:12.033550+0000","last_became_active":"2024-04-22T06:31:11.047278+0000","last_became_peered":"2024-04-22T06:31:11.047278+0000","last_unstale":"2024-04-22T06:31:12.033550+0000","last_undegraded":"2024-04-22T06:31:12.033550+0000","last_fullsized":"2024-04-22T06:31:12.033550+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:32:56.201059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561171+0000","last_change":"2024-04-22T06:31:07.021250+0000","last_active":"2024-04-22T06:31:12.561171+0000","last_peered":"2024-04-22T06:31:12.561171+0000","last_clean":"2024-04-22T06:31:12.561171+0000","last_became_active":"2024-04-22T06:31:07.020876+0000","last_became_peered":"2024-04-22T06:31:07.020876+0000","last_unstale":"2024-04-22T06:31:12.561171+0000","last_undegraded":"2024-04-22T06:31:12.561171+0000","last_fullsized":"2024-04-22T06:31:12.561171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:21:31.210819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561332+0000","last_change":"2024-04-22T06:31:06.012775+0000","last_active":"2024-04-22T06:31:12.561332+0000","last_peered":"2024-04-22T06:31:12.561332+0000","last_clean":"2024-04-22T06:31:12.561332+0000","last_became_active":"2024-04-22T06:31:06.012474+0000","last_became_peered":"2024-04-22T06:31:06.012474+0000","last_unstale":"2024-04-22T06:31:12.561332+0000","last_undegraded":"2024-04-22T06:31:12.561332+0000","last_fullsized":"2024-04-22T06:31:12.561332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:09:48.197011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.033219+0000","last_change":"2024-04-22T06:31:11.057641+0000","last_active":"2024-04-22T06:31:12.033219+0000","last_peered":"2024-04-22T06:31:12.033219+0000","last_clean":"2024-04-22T06:31:12.033219+0000","last_became_active":"2024-04-22T06:31:11.057221+0000","last_became_peered":"2024-04-22T06:31:11.057221+0000","last_unstale":"2024-04-22T06:31:12.033219+0000","last_undegraded":"2024-04-22T06:31:12.033219+0000","last_fullsized":"2024-04-22T06:31:12.033219+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:14:57.713113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559730+0000","last_change":"2024-04-22T06:31:07.020822+0000","last_active":"2024-04-22T06:31:12.559730+0000","last_peered":"2024-04-22T06:31:12.559730+0000","last_clean":"2024-04-22T06:31:12.559730+0000","last_became_active":"2024-04-22T06:31:07.020484+0000","last_became_peered":"2024-04-22T06:31:07.020484+0000","last_unstale":"2024-04-22T06:31:12.559730+0000","last_undegraded":"2024-04-22T06:31:12.559730+0000","last_fullsized":"2024-04-22T06:31:12.559730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:18:26.522131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"25'3","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:37.096092+0000","last_change":"2024-04-22T06:31:06.000000+0000","last_active":"2024-04-22T06:31:37.096092+0000","last_peered":"2024-04-22T06:31:37.096092+0000","last_clean":"2024-04-22T06:31:37.096092+0000","last_became_active":"2024-04-22T06:31:05.999663+0000","last_became_peered":"2024-04-22T06:31:05.999663+0000","last_unstale":"2024-04-22T06:31:37.096092+0000","last_undegraded":"2024-04-22T06:31:37.096092+0000","last_fullsized":"2024-04-22T06:31:37.096092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:42.805084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560901+0000","last_change":"2024-04-22T06:31:11.057991+0000","last_active":"2024-04-22T06:31:12.560901+0000","last_peered":"2024-04-22T06:31:12.560901+0000","last_clean":"2024-04-22T06:31:12.560901+0000","last_became_active":"2024-04-22T06:31:11.057595+0000","last_became_peered":"2024-04-22T06:31:11.057595+0000","last_unstale":"2024-04-22T06:31:12.560901+0000","last_undegraded":"2024-04-22T06:31:12.560901+0000","last_fullsized":"2024-04-22T06:31:12.560901+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:17:06.366663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.033006+0000","last_change":"2024-04-22T06:31:07.017303+0000","last_active":"2024-04-22T06:31:12.033006+0000","last_peered":"2024-04-22T06:31:12.033006+0000","last_clean":"2024-04-22T06:31:12.033006+0000","last_became_active":"2024-04-22T06:31:07.016990+0000","last_became_peered":"2024-04-22T06:31:07.016990+0000","last_unstale":"2024-04-22T06:31:12.033006+0000","last_undegraded":"2024-04-22T06:31:12.033006+0000","last_fullsized":"2024-04-22T06:31:12.033006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:51:38.275472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"25'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:37.091829+0000","last_change":"2024-04-22T06:31:05.997363+0000","last_active":"2024-04-22T06:31:37.091829+0000","last_peered":"2024-04-22T06:31:37.091829+0000","last_clean":"2024-04-22T06:31:37.091829+0000","last_became_active":"2024-04-22T06:31:05.996982+0000","last_became_peered":"2024-04-22T06:31:05.996982+0000","last_unstale":"2024-04-22T06:31:37.091829+0000","last_undegraded":"2024-04-22T06:31:37.091829+0000","last_fullsized":"2024-04-22T06:31:37.091829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:23:50.052130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.033517+0000","last_change":"2024-04-22T06:31:11.048507+0000","last_active":"2024-04-22T06:31:12.033517+0000","last_peered":"2024-04-22T06:31:12.033517+0000","last_clean":"2024-04-22T06:31:12.033517+0000","last_became_active":"2024-04-22T06:31:11.047722+0000","last_became_peered":"2024-04-22T06:31:11.047722+0000","last_unstale":"2024-04-22T06:31:12.033517+0000","last_undegraded":"2024-04-22T06:31:12.033517+0000","last_fullsized":"2024-04-22T06:31:12.033517+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:04:48.732029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561139+0000","last_change":"2024-04-22T06:31:07.021095+0000","last_active":"2024-04-22T06:31:12.561139+0000","last_peered":"2024-04-22T06:31:12.561139+0000","last_clean":"2024-04-22T06:31:12.561139+0000","last_became_active":"2024-04-22T06:31:07.020814+0000","last_became_peered":"2024-04-22T06:31:07.020814+0000","last_unstale":"2024-04-22T06:31:12.561139+0000","last_undegraded":"2024-04-22T06:31:12.561139+0000","last_fullsized":"2024-04-22T06:31:12.561139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:27:08.869392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"25'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:37.089072+0000","last_change":"2024-04-22T06:31:05.997508+0000","last_active":"2024-04-22T06:31:37.089072+0000","last_peered":"2024-04-22T06:31:37.089072+0000","last_clean":"2024-04-22T06:31:37.089072+0000","last_became_active":"2024-04-22T06:31:05.997233+0000","last_became_peered":"2024-04-22T06:31:05.997233+0000","last_unstale":"2024-04-22T06:31:37.089072+0000","last_undegraded":"2024-04-22T06:31:37.089072+0000","last_fullsized":"2024-04-22T06:31:37.089072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:40:56.909549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.032801+0000","last_change":"2024-04-22T06:31:11.045424+0000","last_active":"2024-04-22T06:31:12.032801+0000","last_peered":"2024-04-22T06:31:12.032801+0000","last_clean":"2024-04-22T06:31:12.032801+0000","last_became_active":"2024-04-22T06:31:11.045061+0000","last_became_peered":"2024-04-22T06:31:11.045061+0000","last_unstale":"2024-04-22T06:31:12.032801+0000","last_undegraded":"2024-04-22T06:31:12.032801+0000","last_fullsized":"2024-04-22T06:31:12.032801+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:00:33.958921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559690+0000","last_change":"2024-04-22T06:31:07.026506+0000","last_active":"2024-04-22T06:31:12.559690+0000","last_peered":"2024-04-22T06:31:12.559690+0000","last_clean":"2024-04-22T06:31:12.559690+0000","last_became_active":"2024-04-22T06:31:07.026174+0000","last_became_peered":"2024-04-22T06:31:07.026174+0000","last_unstale":"2024-04-22T06:31:12.559690+0000","last_undegraded":"2024-04-22T06:31:12.559690+0000","last_fullsized":"2024-04-22T06:31:12.559690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:35:21.002170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560793+0000","last_change":"2024-04-22T06:31:05.997990+0000","last_active":"2024-04-22T06:31:12.560793+0000","last_peered":"2024-04-22T06:31:12.560793+0000","last_clean":"2024-04-22T06:31:12.560793+0000","last_became_active":"2024-04-22T06:31:05.997352+0000","last_became_peered":"2024-04-22T06:31:05.997352+0000","last_unstale":"2024-04-22T06:31:12.560793+0000","last_undegraded":"2024-04-22T06:31:12.560793+0000","last_fullsized":"2024-04-22T06:31:12.560793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:15:40.726807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559303+0000","last_change":"2024-04-22T06:31:11.053487+0000","last_active":"2024-04-22T06:31:12.559303+0000","last_peered":"2024-04-22T06:31:12.559303+0000","last_clean":"2024-04-22T06:31:12.559303+0000","last_became_active":"2024-04-22T06:31:11.053078+0000","last_became_peered":"2024-04-22T06:31:11.053078+0000","last_unstale":"2024-04-22T06:31:12.559303+0000","last_undegraded":"2024-04-22T06:31:12.559303+0000","last_fullsized":"2024-04-22T06:31:12.559303+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:23:16.130369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561518+0000","last_change":"2024-04-22T06:31:07.020420+0000","last_active":"2024-04-22T06:31:12.561518+0000","last_peered":"2024-04-22T06:31:12.561518+0000","last_clean":"2024-04-22T06:31:12.561518+0000","last_became_active":"2024-04-22T06:31:07.020110+0000","last_became_peered":"2024-04-22T06:31:07.020110+0000","last_unstale":"2024-04-22T06:31:12.561518+0000","last_undegraded":"2024-04-22T06:31:12.561518+0000","last_fullsized":"2024-04-22T06:31:12.561518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:44:23.997057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"25'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031781+0000","last_change":"2024-04-22T06:31:05.997357+0000","last_active":"2024-04-22T06:31:12.031781+0000","last_peered":"2024-04-22T06:31:12.031781+0000","last_clean":"2024-04-22T06:31:12.031781+0000","last_became_active":"2024-04-22T06:31:05.996913+0000","last_became_peered":"2024-04-22T06:31:05.996913+0000","last_unstale":"2024-04-22T06:31:12.031781+0000","last_undegraded":"2024-04-22T06:31:12.031781+0000","last_fullsized":"2024-04-22T06:31:12.031781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:50:51.880199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560195+0000","last_change":"2024-04-22T06:31:11.059735+0000","last_active":"2024-04-22T06:31:12.560195+0000","last_peered":"2024-04-22T06:31:12.560195+0000","last_clean":"2024-04-22T06:31:12.560195+0000","last_became_active":"2024-04-22T06:31:11.059325+0000","last_became_peered":"2024-04-22T06:31:11.059325+0000","last_unstale":"2024-04-22T06:31:12.560195+0000","last_undegraded":"2024-04-22T06:31:12.560195+0000","last_fullsized":"2024-04-22T06:31:12.560195+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:59:52.881783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.032101+0000","last_change":"2024-04-22T06:31:07.026576+0000","last_active":"2024-04-22T06:31:12.032101+0000","last_peered":"2024-04-22T06:31:12.032101+0000","last_clean":"2024-04-22T06:31:12.032101+0000","last_became_active":"2024-04-22T06:31:07.026057+0000","last_became_peered":"2024-04-22T06:31:07.026057+0000","last_unstale":"2024-04-22T06:31:12.032101+0000","last_undegraded":"2024-04-22T06:31:12.032101+0000","last_fullsized":"2024-04-22T06:31:12.032101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:49:32.525918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561588+0000","last_change":"2024-04-22T06:31:06.009043+0000","last_active":"2024-04-22T06:31:12.561588+0000","last_peered":"2024-04-22T06:31:12.561588+0000","last_clean":"2024-04-22T06:31:12.561588+0000","last_became_active":"2024-04-22T06:31:06.008752+0000","last_became_peered":"2024-04-22T06:31:06.008752+0000","last_unstale":"2024-04-22T06:31:12.561588+0000","last_undegraded":"2024-04-22T06:31:12.561588+0000","last_fullsized":"2024-04-22T06:31:12.561588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:52:34.706288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561059+0000","last_change":"2024-04-22T06:31:11.057688+0000","last_active":"2024-04-22T06:31:12.561059+0000","last_peered":"2024-04-22T06:31:12.561059+0000","last_clean":"2024-04-22T06:31:12.561059+0000","last_became_active":"2024-04-22T06:31:11.057354+0000","last_became_peered":"2024-04-22T06:31:11.057354+0000","last_unstale":"2024-04-22T06:31:12.561059+0000","last_undegraded":"2024-04-22T06:31:12.561059+0000","last_fullsized":"2024-04-22T06:31:12.561059+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:08:36.080352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.033017+0000","last_change":"2024-04-22T06:31:07.028506+0000","last_active":"2024-04-22T06:31:12.033017+0000","last_peered":"2024-04-22T06:31:12.033017+0000","last_clean":"2024-04-22T06:31:12.033017+0000","last_became_active":"2024-04-22T06:31:07.028183+0000","last_became_peered":"2024-04-22T06:31:07.028183+0000","last_unstale":"2024-04-22T06:31:12.033017+0000","last_undegraded":"2024-04-22T06:31:12.033017+0000","last_fullsized":"2024-04-22T06:31:12.033017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:38:45.965934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031683+0000","last_change":"2024-04-22T06:31:06.002052+0000","last_active":"2024-04-22T06:31:12.031683+0000","last_peered":"2024-04-22T06:31:12.031683+0000","last_clean":"2024-04-22T06:31:12.031683+0000","last_became_active":"2024-04-22T06:31:06.000645+0000","last_became_peered":"2024-04-22T06:31:06.000645+0000","last_unstale":"2024-04-22T06:31:12.031683+0000","last_undegraded":"2024-04-22T06:31:12.031683+0000","last_fullsized":"2024-04-22T06:31:12.031683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:55:38.982563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561693+0000","last_change":"2024-04-22T06:31:11.048333+0000","last_active":"2024-04-22T06:31:12.561693+0000","last_peered":"2024-04-22T06:31:12.561693+0000","last_clean":"2024-04-22T06:31:12.561693+0000","last_became_active":"2024-04-22T06:31:11.047913+0000","last_became_peered":"2024-04-22T06:31:11.047913+0000","last_unstale":"2024-04-22T06:31:12.561693+0000","last_undegraded":"2024-04-22T06:31:12.561693+0000","last_fullsized":"2024-04-22T06:31:12.561693+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:11:52.622436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561265+0000","last_change":"2024-04-22T06:31:07.017430+0000","last_active":"2024-04-22T06:31:12.561265+0000","last_peered":"2024-04-22T06:31:12.561265+0000","last_clean":"2024-04-22T06:31:12.561265+0000","last_became_active":"2024-04-22T06:31:07.017112+0000","last_became_peered":"2024-04-22T06:31:07.017112+0000","last_unstale":"2024-04-22T06:31:12.561265+0000","last_undegraded":"2024-04-22T06:31:12.561265+0000","last_fullsized":"2024-04-22T06:31:12.561265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:52:18.098301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"25'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558899+0000","last_change":"2024-04-22T06:31:06.002202+0000","last_active":"2024-04-22T06:31:12.558899+0000","last_peered":"2024-04-22T06:31:12.558899+0000","last_clean":"2024-04-22T06:31:12.558899+0000","last_became_active":"2024-04-22T06:31:06.001916+0000","last_became_peered":"2024-04-22T06:31:06.001916+0000","last_unstale":"2024-04-22T06:31:12.558899+0000","last_undegraded":"2024-04-22T06:31:12.558899+0000","last_fullsized":"2024-04-22T06:31:12.558899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:35:52.485268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560894+0000","last_change":"2024-04-22T06:31:11.054576+0000","last_active":"2024-04-22T06:31:12.560894+0000","last_peered":"2024-04-22T06:31:12.560894+0000","last_clean":"2024-04-22T06:31:12.560894+0000","last_became_active":"2024-04-22T06:31:11.054158+0000","last_became_peered":"2024-04-22T06:31:11.054158+0000","last_unstale":"2024-04-22T06:31:12.560894+0000","last_undegraded":"2024-04-22T06:31:12.560894+0000","last_fullsized":"2024-04-22T06:31:12.560894+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:48:50.809440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558837+0000","last_change":"2024-04-22T06:31:07.023064+0000","last_active":"2024-04-22T06:31:12.558837+0000","last_peered":"2024-04-22T06:31:12.558837+0000","last_clean":"2024-04-22T06:31:12.558837+0000","last_became_active":"2024-04-22T06:31:07.022753+0000","last_became_peered":"2024-04-22T06:31:07.022753+0000","last_unstale":"2024-04-22T06:31:12.558837+0000","last_undegraded":"2024-04-22T06:31:12.558837+0000","last_fullsized":"2024-04-22T06:31:12.558837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:25:38.039689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"25'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.029986+0000","last_change":"2024-04-22T06:31:05.998384+0000","last_active":"2024-04-22T06:31:12.029986+0000","last_peered":"2024-04-22T06:31:12.029986+0000","last_clean":"2024-04-22T06:31:12.029986+0000","last_became_active":"2024-04-22T06:31:05.997761+0000","last_became_peered":"2024-04-22T06:31:05.997761+0000","last_unstale":"2024-04-22T06:31:12.029986+0000","last_undegraded":"2024-04-22T06:31:12.029986+0000","last_fullsized":"2024-04-22T06:31:12.029986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:07:12.014505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561016+0000","last_change":"2024-04-22T06:31:11.060877+0000","last_active":"2024-04-22T06:31:12.561016+0000","last_peered":"2024-04-22T06:31:12.561016+0000","last_clean":"2024-04-22T06:31:12.561016+0000","last_became_active":"2024-04-22T06:31:11.060473+0000","last_became_peered":"2024-04-22T06:31:11.060473+0000","last_unstale":"2024-04-22T06:31:12.561016+0000","last_undegraded":"2024-04-22T06:31:12.561016+0000","last_fullsized":"2024-04-22T06:31:12.561016+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:47:18.456791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558817+0000","last_change":"2024-04-22T06:31:07.026139+0000","last_active":"2024-04-22T06:31:12.558817+0000","last_peered":"2024-04-22T06:31:12.558817+0000","last_clean":"2024-04-22T06:31:12.558817+0000","last_became_active":"2024-04-22T06:31:07.025826+0000","last_became_peered":"2024-04-22T06:31:07.025826+0000","last_unstale":"2024-04-22T06:31:12.558817+0000","last_undegraded":"2024-04-22T06:31:12.558817+0000","last_fullsized":"2024-04-22T06:31:12.558817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:58:48.200581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"25'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:37.095920+0000","last_change":"2024-04-22T06:31:06.001815+0000","last_active":"2024-04-22T06:31:37.095920+0000","last_peered":"2024-04-22T06:31:37.095920+0000","last_clean":"2024-04-22T06:31:37.095920+0000","last_became_active":"2024-04-22T06:31:06.000629+0000","last_became_peered":"2024-04-22T06:31:06.000629+0000","last_unstale":"2024-04-22T06:31:37.095920+0000","last_undegraded":"2024-04-22T06:31:37.095920+0000","last_fullsized":"2024-04-22T06:31:37.095920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:49:31.100969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030036+0000","last_change":"2024-04-22T06:30:56.214982+0000","last_active":"2024-04-22T06:31:12.030036+0000","last_peered":"2024-04-22T06:31:12.030036+0000","last_clean":"2024-04-22T06:31:12.030036+0000","last_became_active":"2024-04-22T06:30:56.214024+0000","last_became_peered":"2024-04-22T06:30:56.214024+0000","last_unstale":"2024-04-22T06:31:12.030036+0000","last_undegraded":"2024-04-22T06:31:12.030036+0000","last_fullsized":"2024-04-22T06:31:12.030036+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:30:54.947156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:30:54.947156+0000","last_clean_scrub_stamp":"2024-04-22T06:30:54.947156+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:32:37.920238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558994+0000","last_change":"2024-04-22T06:31:11.040152+0000","last_active":"2024-04-22T06:31:12.558994+0000","last_peered":"2024-04-22T06:31:12.558994+0000","last_clean":"2024-04-22T06:31:12.558994+0000","last_became_active":"2024-04-22T06:31:11.039749+0000","last_became_peered":"2024-04-22T06:31:11.039749+0000","last_unstale":"2024-04-22T06:31:12.558994+0000","last_undegraded":"2024-04-22T06:31:12.558994+0000","last_fullsized":"2024-04-22T06:31:12.558994+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:31:23.103876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031283+0000","last_change":"2024-04-22T06:31:07.011062+0000","last_active":"2024-04-22T06:31:12.031283+0000","last_peered":"2024-04-22T06:31:12.031283+0000","last_clean":"2024-04-22T06:31:12.031283+0000","last_became_active":"2024-04-22T06:31:07.010755+0000","last_became_peered":"2024-04-22T06:31:07.010755+0000","last_unstale":"2024-04-22T06:31:12.031283+0000","last_undegraded":"2024-04-22T06:31:12.031283+0000","last_fullsized":"2024-04-22T06:31:12.031283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:57:01.373036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560069+0000","last_change":"2024-04-22T06:31:05.998083+0000","last_active":"2024-04-22T06:31:12.560069+0000","last_peered":"2024-04-22T06:31:12.560069+0000","last_clean":"2024-04-22T06:31:12.560069+0000","last_became_active":"2024-04-22T06:31:05.997777+0000","last_became_peered":"2024-04-22T06:31:05.997777+0000","last_unstale":"2024-04-22T06:31:12.560069+0000","last_undegraded":"2024-04-22T06:31:12.560069+0000","last_fullsized":"2024-04-22T06:31:12.560069+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:15:13.171198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558666+0000","last_change":"2024-04-22T06:31:11.049607+0000","last_active":"2024-04-22T06:31:12.558666+0000","last_peered":"2024-04-22T06:31:12.558666+0000","last_clean":"2024-04-22T06:31:12.558666+0000","last_became_active":"2024-04-22T06:31:11.049156+0000","last_became_peered":"2024-04-22T06:31:11.049156+0000","last_unstale":"2024-04-22T06:31:12.558666+0000","last_undegraded":"2024-04-22T06:31:12.558666+0000","last_fullsized":"2024-04-22T06:31:12.558666+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:15:48.599278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.032032+0000","last_change":"2024-04-22T06:31:07.026306+0000","last_active":"2024-04-22T06:31:12.032032+0000","last_peered":"2024-04-22T06:31:12.032032+0000","last_clean":"2024-04-22T06:31:12.032032+0000","last_became_active":"2024-04-22T06:31:07.025987+0000","last_became_peered":"2024-04-22T06:31:07.025987+0000","last_unstale":"2024-04-22T06:31:12.032032+0000","last_undegraded":"2024-04-22T06:31:12.032032+0000","last_fullsized":"2024-04-22T06:31:12.032032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:50:31.803180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560942+0000","last_change":"2024-04-22T06:31:06.003115+0000","last_active":"2024-04-22T06:31:12.560942+0000","last_peered":"2024-04-22T06:31:12.560942+0000","last_clean":"2024-04-22T06:31:12.560942+0000","last_became_active":"2024-04-22T06:31:06.001945+0000","last_became_peered":"2024-04-22T06:31:06.001945+0000","last_unstale":"2024-04-22T06:31:12.560942+0000","last_undegraded":"2024-04-22T06:31:12.560942+0000","last_fullsized":"2024-04-22T06:31:12.560942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:59:25.770504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561105+0000","last_change":"2024-04-22T06:31:11.058841+0000","last_active":"2024-04-22T06:31:12.561105+0000","last_peered":"2024-04-22T06:31:12.561105+0000","last_clean":"2024-04-22T06:31:12.561105+0000","last_became_active":"2024-04-22T06:31:11.058407+0000","last_became_peered":"2024-04-22T06:31:11.058407+0000","last_unstale":"2024-04-22T06:31:12.561105+0000","last_undegraded":"2024-04-22T06:31:12.561105+0000","last_fullsized":"2024-04-22T06:31:12.561105+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:30:20.432210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031032+0000","last_change":"2024-04-22T06:31:07.022314+0000","last_active":"2024-04-22T06:31:12.031032+0000","last_peered":"2024-04-22T06:31:12.031032+0000","last_clean":"2024-04-22T06:31:12.031032+0000","last_became_active":"2024-04-22T06:31:07.021497+0000","last_became_peered":"2024-04-22T06:31:07.021497+0000","last_unstale":"2024-04-22T06:31:12.031032+0000","last_undegraded":"2024-04-22T06:31:12.031032+0000","last_fullsized":"2024-04-22T06:31:12.031032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:16:25.960359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559348+0000","last_change":"2024-04-22T06:31:06.003805+0000","last_active":"2024-04-22T06:31:12.559348+0000","last_peered":"2024-04-22T06:31:12.559348+0000","last_clean":"2024-04-22T06:31:12.559348+0000","last_became_active":"2024-04-22T06:31:06.002735+0000","last_became_peered":"2024-04-22T06:31:06.002735+0000","last_unstale":"2024-04-22T06:31:12.559348+0000","last_undegraded":"2024-04-22T06:31:12.559348+0000","last_fullsized":"2024-04-22T06:31:12.559348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:41:16.155628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030380+0000","last_change":"2024-04-22T06:31:11.043003+0000","last_active":"2024-04-22T06:31:12.030380+0000","last_peered":"2024-04-22T06:31:12.030380+0000","last_clean":"2024-04-22T06:31:12.030380+0000","last_became_active":"2024-04-22T06:31:11.042599+0000","last_became_peered":"2024-04-22T06:31:11.042599+0000","last_unstale":"2024-04-22T06:31:12.030380+0000","last_undegraded":"2024-04-22T06:31:12.030380+0000","last_fullsized":"2024-04-22T06:31:12.030380+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:28:58.949214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030520+0000","last_change":"2024-04-22T06:31:07.021258+0000","last_active":"2024-04-22T06:31:12.030520+0000","last_peered":"2024-04-22T06:31:12.030520+0000","last_clean":"2024-04-22T06:31:12.030520+0000","last_became_active":"2024-04-22T06:31:07.020976+0000","last_became_peered":"2024-04-22T06:31:07.020976+0000","last_unstale":"2024-04-22T06:31:12.030520+0000","last_undegraded":"2024-04-22T06:31:12.030520+0000","last_fullsized":"2024-04-22T06:31:12.030520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:19:33.377626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"24'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.033565+0000","last_change":"2024-04-22T06:31:06.002751+0000","last_active":"2024-04-22T06:31:12.033565+0000","last_peered":"2024-04-22T06:31:12.033565+0000","last_clean":"2024-04-22T06:31:12.033565+0000","last_became_active":"2024-04-22T06:31:06.002489+0000","last_became_peered":"2024-04-22T06:31:06.002489+0000","last_unstale":"2024-04-22T06:31:12.033565+0000","last_undegraded":"2024-04-22T06:31:12.033565+0000","last_fullsized":"2024-04-22T06:31:12.033565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:20:05.937529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.032720+0000","last_change":"2024-04-22T06:31:11.048954+0000","last_active":"2024-04-22T06:31:12.032720+0000","last_peered":"2024-04-22T06:31:12.032720+0000","last_clean":"2024-04-22T06:31:12.032720+0000","last_became_active":"2024-04-22T06:31:11.048214+0000","last_became_peered":"2024-04-22T06:31:11.048214+0000","last_unstale":"2024-04-22T06:31:12.032720+0000","last_undegraded":"2024-04-22T06:31:12.032720+0000","last_fullsized":"2024-04-22T06:31:12.032720+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:13:21.903203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030103+0000","last_change":"2024-04-22T06:31:07.021258+0000","last_active":"2024-04-22T06:31:12.030103+0000","last_peered":"2024-04-22T06:31:12.030103+0000","last_clean":"2024-04-22T06:31:12.030103+0000","last_became_active":"2024-04-22T06:31:07.021036+0000","last_became_peered":"2024-04-22T06:31:07.021036+0000","last_unstale":"2024-04-22T06:31:12.030103+0000","last_undegraded":"2024-04-22T06:31:12.030103+0000","last_fullsized":"2024-04-22T06:31:12.030103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:51:39.990228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"25'10","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:37.093513+0000","last_change":"2024-04-22T06:31:05.998270+0000","last_active":"2024-04-22T06:31:37.093513+0000","last_peered":"2024-04-22T06:31:37.093513+0000","last_clean":"2024-04-22T06:31:37.093513+0000","last_became_active":"2024-04-22T06:31:05.997925+0000","last_became_peered":"2024-04-22T06:31:05.997925+0000","last_unstale":"2024-04-22T06:31:37.093513+0000","last_undegraded":"2024-04-22T06:31:37.093513+0000","last_fullsized":"2024-04-22T06:31:37.093513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:49:44.167932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560845+0000","last_change":"2024-04-22T06:31:11.051106+0000","last_active":"2024-04-22T06:31:12.560845+0000","last_peered":"2024-04-22T06:31:12.560845+0000","last_clean":"2024-04-22T06:31:12.560845+0000","last_became_active":"2024-04-22T06:31:11.050721+0000","last_became_peered":"2024-04-22T06:31:11.050721+0000","last_unstale":"2024-04-22T06:31:12.560845+0000","last_undegraded":"2024-04-22T06:31:12.560845+0000","last_fullsized":"2024-04-22T06:31:12.560845+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:58:43.521850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558584+0000","last_change":"2024-04-22T06:31:07.020281+0000","last_active":"2024-04-22T06:31:12.558584+0000","last_peered":"2024-04-22T06:31:12.558584+0000","last_clean":"2024-04-22T06:31:12.558584+0000","last_became_active":"2024-04-22T06:31:07.020012+0000","last_became_peered":"2024-04-22T06:31:07.020012+0000","last_unstale":"2024-04-22T06:31:12.558584+0000","last_undegraded":"2024-04-22T06:31:12.558584+0000","last_fullsized":"2024-04-22T06:31:12.558584+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:20:40.301631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558743+0000","last_change":"2024-04-22T06:31:05.999918+0000","last_active":"2024-04-22T06:31:12.558743+0000","last_peered":"2024-04-22T06:31:12.558743+0000","last_clean":"2024-04-22T06:31:12.558743+0000","last_became_active":"2024-04-22T06:31:05.999594+0000","last_became_peered":"2024-04-22T06:31:05.999594+0000","last_unstale":"2024-04-22T06:31:12.558743+0000","last_undegraded":"2024-04-22T06:31:12.558743+0000","last_fullsized":"2024-04-22T06:31:12.558743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:15:21.376196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559535+0000","last_change":"2024-04-22T06:31:11.055940+0000","last_active":"2024-04-22T06:31:12.559535+0000","last_peered":"2024-04-22T06:31:12.559535+0000","last_clean":"2024-04-22T06:31:12.559535+0000","last_became_active":"2024-04-22T06:31:11.054873+0000","last_became_peered":"2024-04-22T06:31:11.054873+0000","last_unstale":"2024-04-22T06:31:12.559535+0000","last_undegraded":"2024-04-22T06:31:12.559535+0000","last_fullsized":"2024-04-22T06:31:12.559535+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:13:51.061199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030857+0000","last_change":"2024-04-22T06:31:07.021839+0000","last_active":"2024-04-22T06:31:12.030857+0000","last_peered":"2024-04-22T06:31:12.030857+0000","last_clean":"2024-04-22T06:31:12.030857+0000","last_became_active":"2024-04-22T06:31:07.021585+0000","last_became_peered":"2024-04-22T06:31:07.021585+0000","last_unstale":"2024-04-22T06:31:12.030857+0000","last_undegraded":"2024-04-22T06:31:12.030857+0000","last_fullsized":"2024-04-22T06:31:12.030857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:55:15.275802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"25'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:37.093627+0000","last_change":"2024-04-22T06:31:06.004058+0000","last_active":"2024-04-22T06:31:37.093627+0000","last_peered":"2024-04-22T06:31:37.093627+0000","last_clean":"2024-04-22T06:31:37.093627+0000","last_became_active":"2024-04-22T06:31:06.003791+0000","last_became_peered":"2024-04-22T06:31:06.003791+0000","last_unstale":"2024-04-22T06:31:37.093627+0000","last_undegraded":"2024-04-22T06:31:37.093627+0000","last_fullsized":"2024-04-22T06:31:37.093627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:03:08.913113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559058+0000","last_change":"2024-04-22T06:31:11.054245+0000","last_active":"2024-04-22T06:31:12.559058+0000","last_peered":"2024-04-22T06:31:12.559058+0000","last_clean":"2024-04-22T06:31:12.559058+0000","last_became_active":"2024-04-22T06:31:11.053551+0000","last_became_peered":"2024-04-22T06:31:11.053551+0000","last_unstale":"2024-04-22T06:31:12.559058+0000","last_undegraded":"2024-04-22T06:31:12.559058+0000","last_fullsized":"2024-04-22T06:31:12.559058+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:34:06.312700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559478+0000","last_change":"2024-04-22T06:31:07.023273+0000","last_active":"2024-04-22T06:31:12.559478+0000","last_peered":"2024-04-22T06:31:12.559478+0000","last_clean":"2024-04-22T06:31:12.559478+0000","last_became_active":"2024-04-22T06:31:07.022978+0000","last_became_peered":"2024-04-22T06:31:07.022978+0000","last_unstale":"2024-04-22T06:31:12.559478+0000","last_undegraded":"2024-04-22T06:31:12.559478+0000","last_fullsized":"2024-04-22T06:31:12.559478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:10:50.192298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"25'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:37.091741+0000","last_change":"2024-04-22T06:31:06.005510+0000","last_active":"2024-04-22T06:31:37.091741+0000","last_peered":"2024-04-22T06:31:37.091741+0000","last_clean":"2024-04-22T06:31:37.091741+0000","last_became_active":"2024-04-22T06:31:06.005047+0000","last_became_peered":"2024-04-22T06:31:06.005047+0000","last_unstale":"2024-04-22T06:31:37.091741+0000","last_undegraded":"2024-04-22T06:31:37.091741+0000","last_fullsized":"2024-04-22T06:31:37.091741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:25:51.227631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.032100+0000","last_change":"2024-04-22T06:31:11.046209+0000","last_active":"2024-04-22T06:31:12.032100+0000","last_peered":"2024-04-22T06:31:12.032100+0000","last_clean":"2024-04-22T06:31:12.032100+0000","last_became_active":"2024-04-22T06:31:11.045339+0000","last_became_peered":"2024-04-22T06:31:11.045339+0000","last_unstale":"2024-04-22T06:31:12.032100+0000","last_undegraded":"2024-04-22T06:31:12.032100+0000","last_fullsized":"2024-04-22T06:31:12.032100+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:39:16.659010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031172+0000","last_change":"2024-04-22T06:31:07.025901+0000","last_active":"2024-04-22T06:31:12.031172+0000","last_peered":"2024-04-22T06:31:12.031172+0000","last_clean":"2024-04-22T06:31:12.031172+0000","last_became_active":"2024-04-22T06:31:07.025590+0000","last_became_peered":"2024-04-22T06:31:07.025590+0000","last_unstale":"2024-04-22T06:31:12.031172+0000","last_undegraded":"2024-04-22T06:31:12.031172+0000","last_fullsized":"2024-04-22T06:31:12.031172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:00:26.184054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'11","reported_seq":33,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:37.094322+0000","last_change":"2024-04-22T06:31:06.004609+0000","last_active":"2024-04-22T06:31:37.094322+0000","last_peered":"2024-04-22T06:31:37.094322+0000","last_clean":"2024-04-22T06:31:37.094322+0000","last_became_active":"2024-04-22T06:31:06.004319+0000","last_became_peered":"2024-04-22T06:31:06.004319+0000","last_unstale":"2024-04-22T06:31:37.094322+0000","last_undegraded":"2024-04-22T06:31:37.094322+0000","last_fullsized":"2024-04-22T06:31:37.094322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:32:47.198906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561266+0000","last_change":"2024-04-22T06:31:11.054706+0000","last_active":"2024-04-22T06:31:12.561266+0000","last_peered":"2024-04-22T06:31:12.561266+0000","last_clean":"2024-04-22T06:31:12.561266+0000","last_became_active":"2024-04-22T06:31:11.054307+0000","last_became_peered":"2024-04-22T06:31:11.054307+0000","last_unstale":"2024-04-22T06:31:12.561266+0000","last_undegraded":"2024-04-22T06:31:12.561266+0000","last_fullsized":"2024-04-22T06:31:12.561266+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:05:02.465114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.033403+0000","last_change":"2024-04-22T06:31:07.020739+0000","last_active":"2024-04-22T06:31:12.033403+0000","last_peered":"2024-04-22T06:31:12.033403+0000","last_clean":"2024-04-22T06:31:12.033403+0000","last_became_active":"2024-04-22T06:31:07.020070+0000","last_became_peered":"2024-04-22T06:31:07.020070+0000","last_unstale":"2024-04-22T06:31:12.033403+0000","last_undegraded":"2024-04-22T06:31:12.033403+0000","last_fullsized":"2024-04-22T06:31:12.033403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:50:22.102753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"25'4","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560870+0000","last_change":"2024-04-22T06:31:06.003733+0000","last_active":"2024-04-22T06:31:12.560870+0000","last_peered":"2024-04-22T06:31:12.560870+0000","last_clean":"2024-04-22T06:31:12.560870+0000","last_became_active":"2024-04-22T06:31:06.003465+0000","last_became_peered":"2024-04-22T06:31:06.003465+0000","last_unstale":"2024-04-22T06:31:12.560870+0000","last_undegraded":"2024-04-22T06:31:12.560870+0000","last_fullsized":"2024-04-22T06:31:12.560870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:19:36.023311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.032875+0000","last_change":"2024-04-22T06:31:11.058041+0000","last_active":"2024-04-22T06:31:12.032875+0000","last_peered":"2024-04-22T06:31:12.032875+0000","last_clean":"2024-04-22T06:31:12.032875+0000","last_became_active":"2024-04-22T06:31:11.057243+0000","last_became_peered":"2024-04-22T06:31:11.057243+0000","last_unstale":"2024-04-22T06:31:12.032875+0000","last_undegraded":"2024-04-22T06:31:12.032875+0000","last_fullsized":"2024-04-22T06:31:12.032875+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:49:06.781140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560767+0000","last_change":"2024-04-22T06:31:07.015369+0000","last_active":"2024-04-22T06:31:12.560767+0000","last_peered":"2024-04-22T06:31:12.560767+0000","last_clean":"2024-04-22T06:31:12.560767+0000","last_became_active":"2024-04-22T06:31:07.015088+0000","last_became_peered":"2024-04-22T06:31:07.015088+0000","last_unstale":"2024-04-22T06:31:12.560767+0000","last_undegraded":"2024-04-22T06:31:12.560767+0000","last_fullsized":"2024-04-22T06:31:12.560767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:10:09.449507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"25'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:37.089606+0000","last_change":"2024-04-22T06:31:05.999636+0000","last_active":"2024-04-22T06:31:37.089606+0000","last_peered":"2024-04-22T06:31:37.089606+0000","last_clean":"2024-04-22T06:31:37.089606+0000","last_became_active":"2024-04-22T06:31:05.999329+0000","last_became_peered":"2024-04-22T06:31:05.999329+0000","last_unstale":"2024-04-22T06:31:37.089606+0000","last_undegraded":"2024-04-22T06:31:37.089606+0000","last_fullsized":"2024-04-22T06:31:37.089606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:09:06.670827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559595+0000","last_change":"2024-04-22T06:31:11.053530+0000","last_active":"2024-04-22T06:31:12.559595+0000","last_peered":"2024-04-22T06:31:12.559595+0000","last_clean":"2024-04-22T06:31:12.559595+0000","last_became_active":"2024-04-22T06:31:11.053100+0000","last_became_peered":"2024-04-22T06:31:11.053100+0000","last_unstale":"2024-04-22T06:31:12.559595+0000","last_undegraded":"2024-04-22T06:31:12.559595+0000","last_fullsized":"2024-04-22T06:31:12.559595+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:57:16.506840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030657+0000","last_change":"2024-04-22T06:31:07.015838+0000","last_active":"2024-04-22T06:31:12.030657+0000","last_peered":"2024-04-22T06:31:12.030657+0000","last_clean":"2024-04-22T06:31:12.030657+0000","last_became_active":"2024-04-22T06:31:07.015545+0000","last_became_peered":"2024-04-22T06:31:07.015545+0000","last_unstale":"2024-04-22T06:31:12.030657+0000","last_undegraded":"2024-04-22T06:31:12.030657+0000","last_fullsized":"2024-04-22T06:31:12.030657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:43:54.251061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"25'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:37.089896+0000","last_change":"2024-04-22T06:31:06.005442+0000","last_active":"2024-04-22T06:31:37.089896+0000","last_peered":"2024-04-22T06:31:37.089896+0000","last_clean":"2024-04-22T06:31:37.089896+0000","last_became_active":"2024-04-22T06:31:06.005156+0000","last_became_peered":"2024-04-22T06:31:06.005156+0000","last_unstale":"2024-04-22T06:31:37.089896+0000","last_undegraded":"2024-04-22T06:31:37.089896+0000","last_fullsized":"2024-04-22T06:31:37.089896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:52:32.138519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560401+0000","last_change":"2024-04-22T06:31:11.046885+0000","last_active":"2024-04-22T06:31:12.560401+0000","last_peered":"2024-04-22T06:31:12.560401+0000","last_clean":"2024-04-22T06:31:12.560401+0000","last_became_active":"2024-04-22T06:31:11.046444+0000","last_became_peered":"2024-04-22T06:31:11.046444+0000","last_unstale":"2024-04-22T06:31:12.560401+0000","last_undegraded":"2024-04-22T06:31:12.560401+0000","last_fullsized":"2024-04-22T06:31:12.560401+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:17:54.523416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560687+0000","last_change":"2024-04-22T06:31:11.061784+0000","last_active":"2024-04-22T06:31:12.560687+0000","last_peered":"2024-04-22T06:31:12.560687+0000","last_clean":"2024-04-22T06:31:12.560687+0000","last_became_active":"2024-04-22T06:31:11.061382+0000","last_became_peered":"2024-04-22T06:31:11.061382+0000","last_unstale":"2024-04-22T06:31:12.560687+0000","last_undegraded":"2024-04-22T06:31:12.560687+0000","last_fullsized":"2024-04-22T06:31:12.560687+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:12:53.829206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560353+0000","last_change":"2024-04-22T06:31:07.023542+0000","last_active":"2024-04-22T06:31:12.560353+0000","last_peered":"2024-04-22T06:31:12.560353+0000","last_clean":"2024-04-22T06:31:12.560353+0000","last_became_active":"2024-04-22T06:31:07.023240+0000","last_became_peered":"2024-04-22T06:31:07.023240+0000","last_unstale":"2024-04-22T06:31:12.560353+0000","last_undegraded":"2024-04-22T06:31:12.560353+0000","last_fullsized":"2024-04-22T06:31:12.560353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:15:34.727195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030715+0000","last_change":"2024-04-22T06:31:06.003080+0000","last_active":"2024-04-22T06:31:12.030715+0000","last_peered":"2024-04-22T06:31:12.030715+0000","last_clean":"2024-04-22T06:31:12.030715+0000","last_became_active":"2024-04-22T06:31:06.002725+0000","last_became_peered":"2024-04-22T06:31:06.002725+0000","last_unstale":"2024-04-22T06:31:12.030715+0000","last_undegraded":"2024-04-22T06:31:12.030715+0000","last_fullsized":"2024-04-22T06:31:12.030715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:46:38.249137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559100+0000","last_change":"2024-04-22T06:31:11.047782+0000","last_active":"2024-04-22T06:31:12.559100+0000","last_peered":"2024-04-22T06:31:12.559100+0000","last_clean":"2024-04-22T06:31:12.559100+0000","last_became_active":"2024-04-22T06:31:11.047407+0000","last_became_peered":"2024-04-22T06:31:11.047407+0000","last_unstale":"2024-04-22T06:31:12.559100+0000","last_undegraded":"2024-04-22T06:31:12.559100+0000","last_fullsized":"2024-04-22T06:31:12.559100+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:48:05.416994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559643+0000","last_change":"2024-04-22T06:31:07.025799+0000","last_active":"2024-04-22T06:31:12.559643+0000","last_peered":"2024-04-22T06:31:12.559643+0000","last_clean":"2024-04-22T06:31:12.559643+0000","last_became_active":"2024-04-22T06:31:07.025420+0000","last_became_peered":"2024-04-22T06:31:07.025420+0000","last_unstale":"2024-04-22T06:31:12.559643+0000","last_undegraded":"2024-04-22T06:31:12.559643+0000","last_fullsized":"2024-04-22T06:31:12.559643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:39:40.107621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560547+0000","last_change":"2024-04-22T06:31:06.005805+0000","last_active":"2024-04-22T06:31:12.560547+0000","last_peered":"2024-04-22T06:31:12.560547+0000","last_clean":"2024-04-22T06:31:12.560547+0000","last_became_active":"2024-04-22T06:31:06.005479+0000","last_became_peered":"2024-04-22T06:31:06.005479+0000","last_unstale":"2024-04-22T06:31:12.560547+0000","last_undegraded":"2024-04-22T06:31:12.560547+0000","last_fullsized":"2024-04-22T06:31:12.560547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:59:37.140612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"24'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558740+0000","last_change":"2024-04-22T06:31:05.998354+0000","last_active":"2024-04-22T06:31:12.558740+0000","last_peered":"2024-04-22T06:31:12.558740+0000","last_clean":"2024-04-22T06:31:12.558740+0000","last_became_active":"2024-04-22T06:31:05.998071+0000","last_became_peered":"2024-04-22T06:31:05.998071+0000","last_unstale":"2024-04-22T06:31:12.558740+0000","last_undegraded":"2024-04-22T06:31:12.558740+0000","last_fullsized":"2024-04-22T06:31:12.558740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:12:13.343435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561078+0000","last_change":"2024-04-22T06:31:07.024109+0000","last_active":"2024-04-22T06:31:12.561078+0000","last_peered":"2024-04-22T06:31:12.561078+0000","last_clean":"2024-04-22T06:31:12.561078+0000","last_became_active":"2024-04-22T06:31:07.023346+0000","last_became_peered":"2024-04-22T06:31:07.023346+0000","last_unstale":"2024-04-22T06:31:12.561078+0000","last_undegraded":"2024-04-22T06:31:12.561078+0000","last_fullsized":"2024-04-22T06:31:12.561078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:02:42.662312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559888+0000","last_change":"2024-04-22T06:31:11.047132+0000","last_active":"2024-04-22T06:31:12.559888+0000","last_peered":"2024-04-22T06:31:12.559888+0000","last_clean":"2024-04-22T06:31:12.559888+0000","last_became_active":"2024-04-22T06:31:11.046671+0000","last_became_peered":"2024-04-22T06:31:11.046671+0000","last_unstale":"2024-04-22T06:31:12.559888+0000","last_undegraded":"2024-04-22T06:31:12.559888+0000","last_fullsized":"2024-04-22T06:31:12.559888+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:41:57.706735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"25'5","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:37.094390+0000","last_change":"2024-04-22T06:31:06.004971+0000","last_active":"2024-04-22T06:31:37.094390+0000","last_peered":"2024-04-22T06:31:37.094390+0000","last_clean":"2024-04-22T06:31:37.094390+0000","last_became_active":"2024-04-22T06:31:06.004473+0000","last_became_peered":"2024-04-22T06:31:06.004473+0000","last_unstale":"2024-04-22T06:31:37.094390+0000","last_undegraded":"2024-04-22T06:31:37.094390+0000","last_fullsized":"2024-04-22T06:31:37.094390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:47:11.735311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.032950+0000","last_change":"2024-04-22T06:31:07.018314+0000","last_active":"2024-04-22T06:31:12.032950+0000","last_peered":"2024-04-22T06:31:12.032950+0000","last_clean":"2024-04-22T06:31:12.032950+0000","last_became_active":"2024-04-22T06:31:07.018033+0000","last_became_peered":"2024-04-22T06:31:07.018033+0000","last_unstale":"2024-04-22T06:31:12.032950+0000","last_undegraded":"2024-04-22T06:31:12.032950+0000","last_fullsized":"2024-04-22T06:31:12.032950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:06:06.099494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559988+0000","last_change":"2024-04-22T06:31:11.055874+0000","last_active":"2024-04-22T06:31:12.559988+0000","last_peered":"2024-04-22T06:31:12.559988+0000","last_clean":"2024-04-22T06:31:12.559988+0000","last_became_active":"2024-04-22T06:31:11.055371+0000","last_became_peered":"2024-04-22T06:31:11.055371+0000","last_unstale":"2024-04-22T06:31:12.559988+0000","last_undegraded":"2024-04-22T06:31:12.559988+0000","last_fullsized":"2024-04-22T06:31:12.559988+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:06:03.909337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558461+0000","last_change":"2024-04-22T06:31:07.024780+0000","last_active":"2024-04-22T06:31:12.558461+0000","last_peered":"2024-04-22T06:31:12.558461+0000","last_clean":"2024-04-22T06:31:12.558461+0000","last_became_active":"2024-04-22T06:31:07.024501+0000","last_became_peered":"2024-04-22T06:31:07.024501+0000","last_unstale":"2024-04-22T06:31:12.558461+0000","last_undegraded":"2024-04-22T06:31:12.558461+0000","last_fullsized":"2024-04-22T06:31:12.558461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:12:45.981232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558747+0000","last_change":"2024-04-22T06:31:07.014546+0000","last_active":"2024-04-22T06:31:12.558747+0000","last_peered":"2024-04-22T06:31:12.558747+0000","last_clean":"2024-04-22T06:31:12.558747+0000","last_became_active":"2024-04-22T06:31:07.014196+0000","last_became_peered":"2024-04-22T06:31:07.014196+0000","last_unstale":"2024-04-22T06:31:12.558747+0000","last_undegraded":"2024-04-22T06:31:12.558747+0000","last_fullsized":"2024-04-22T06:31:12.558747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:57:33.136875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031490+0000","last_change":"2024-04-22T06:31:07.018563+0000","last_active":"2024-04-22T06:31:12.031490+0000","last_peered":"2024-04-22T06:31:12.031490+0000","last_clean":"2024-04-22T06:31:12.031490+0000","last_became_active":"2024-04-22T06:31:07.018289+0000","last_became_peered":"2024-04-22T06:31:07.018289+0000","last_unstale":"2024-04-22T06:31:12.031490+0000","last_undegraded":"2024-04-22T06:31:12.031490+0000","last_fullsized":"2024-04-22T06:31:12.031490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:30:11.892544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560983+0000","last_change":"2024-04-22T06:31:07.022901+0000","last_active":"2024-04-22T06:31:12.560983+0000","last_peered":"2024-04-22T06:31:12.560983+0000","last_clean":"2024-04-22T06:31:12.560983+0000","last_became_active":"2024-04-22T06:31:07.022420+0000","last_became_peered":"2024-04-22T06:31:07.022420+0000","last_unstale":"2024-04-22T06:31:12.560983+0000","last_undegraded":"2024-04-22T06:31:12.560983+0000","last_fullsized":"2024-04-22T06:31:12.560983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:24:18.929896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559253+0000","last_change":"2024-04-22T06:31:07.022875+0000","last_active":"2024-04-22T06:31:12.559253+0000","last_peered":"2024-04-22T06:31:12.559253+0000","last_clean":"2024-04-22T06:31:12.559253+0000","last_became_active":"2024-04-22T06:31:07.022545+0000","last_became_peered":"2024-04-22T06:31:07.022545+0000","last_unstale":"2024-04-22T06:31:12.559253+0000","last_undegraded":"2024-04-22T06:31:12.559253+0000","last_fullsized":"2024-04-22T06:31:12.559253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:02:30.464508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560354+0000","last_change":"2024-04-22T06:31:07.026367+0000","last_active":"2024-04-22T06:31:12.560354+0000","last_peered":"2024-04-22T06:31:12.560354+0000","last_clean":"2024-04-22T06:31:12.560354+0000","last_became_active":"2024-04-22T06:31:07.026085+0000","last_became_peered":"2024-04-22T06:31:07.026085+0000","last_unstale":"2024-04-22T06:31:12.560354+0000","last_undegraded":"2024-04-22T06:31:12.560354+0000","last_fullsized":"2024-04-22T06:31:12.560354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:02:55.368591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559243+0000","last_change":"2024-04-22T06:31:07.022989+0000","last_active":"2024-04-22T06:31:12.559243+0000","last_peered":"2024-04-22T06:31:12.559243+0000","last_clean":"2024-04-22T06:31:12.559243+0000","last_became_active":"2024-04-22T06:31:07.022688+0000","last_became_peered":"2024-04-22T06:31:07.022688+0000","last_unstale":"2024-04-22T06:31:12.559243+0000","last_undegraded":"2024-04-22T06:31:12.559243+0000","last_fullsized":"2024-04-22T06:31:12.559243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:24:10.910731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559590+0000","last_change":"2024-04-22T06:31:07.026315+0000","last_active":"2024-04-22T06:31:12.559590+0000","last_peered":"2024-04-22T06:31:12.559590+0000","last_clean":"2024-04-22T06:31:12.559590+0000","last_became_active":"2024-04-22T06:31:07.025951+0000","last_became_peered":"2024-04-22T06:31:07.025951+0000","last_unstale":"2024-04-22T06:31:12.559590+0000","last_undegraded":"2024-04-22T06:31:12.559590+0000","last_fullsized":"2024-04-22T06:31:12.559590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:54:12.160058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030585+0000","last_change":"2024-04-22T06:31:07.022748+0000","last_active":"2024-04-22T06:31:12.030585+0000","last_peered":"2024-04-22T06:31:12.030585+0000","last_clean":"2024-04-22T06:31:12.030585+0000","last_became_active":"2024-04-22T06:31:07.022481+0000","last_became_peered":"2024-04-22T06:31:07.022481+0000","last_unstale":"2024-04-22T06:31:12.030585+0000","last_undegraded":"2024-04-22T06:31:12.030585+0000","last_fullsized":"2024-04-22T06:31:12.030585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:56:32.631637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560456+0000","last_change":"2024-04-22T06:31:07.021939+0000","last_active":"2024-04-22T06:31:12.560456+0000","last_peered":"2024-04-22T06:31:12.560456+0000","last_clean":"2024-04-22T06:31:12.560456+0000","last_became_active":"2024-04-22T06:31:07.021638+0000","last_became_peered":"2024-04-22T06:31:07.021638+0000","last_unstale":"2024-04-22T06:31:12.560456+0000","last_undegraded":"2024-04-22T06:31:12.560456+0000","last_fullsized":"2024-04-22T06:31:12.560456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:05:27.106092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031390+0000","last_change":"2024-04-22T06:31:07.015844+0000","last_active":"2024-04-22T06:31:12.031390+0000","last_peered":"2024-04-22T06:31:12.031390+0000","last_clean":"2024-04-22T06:31:12.031390+0000","last_became_active":"2024-04-22T06:31:07.015520+0000","last_became_peered":"2024-04-22T06:31:07.015520+0000","last_unstale":"2024-04-22T06:31:12.031390+0000","last_undegraded":"2024-04-22T06:31:12.031390+0000","last_fullsized":"2024-04-22T06:31:12.031390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:24:04.676892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560128+0000","last_change":"2024-04-22T06:31:07.023697+0000","last_active":"2024-04-22T06:31:12.560128+0000","last_peered":"2024-04-22T06:31:12.560128+0000","last_clean":"2024-04-22T06:31:12.560128+0000","last_became_active":"2024-04-22T06:31:07.023283+0000","last_became_peered":"2024-04-22T06:31:07.023283+0000","last_unstale":"2024-04-22T06:31:12.560128+0000","last_undegraded":"2024-04-22T06:31:12.560128+0000","last_fullsized":"2024-04-22T06:31:12.560128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:43:01.338104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558868+0000","last_change":"2024-04-22T06:31:07.023953+0000","last_active":"2024-04-22T06:31:12.558868+0000","last_peered":"2024-04-22T06:31:12.558868+0000","last_clean":"2024-04-22T06:31:12.558868+0000","last_became_active":"2024-04-22T06:31:07.023704+0000","last_became_peered":"2024-04-22T06:31:07.023704+0000","last_unstale":"2024-04-22T06:31:12.558868+0000","last_undegraded":"2024-04-22T06:31:12.558868+0000","last_fullsized":"2024-04-22T06:31:12.558868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:59:32.219492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031619+0000","last_change":"2024-04-22T06:31:07.027418+0000","last_active":"2024-04-22T06:31:12.031619+0000","last_peered":"2024-04-22T06:31:12.031619+0000","last_clean":"2024-04-22T06:31:12.031619+0000","last_became_active":"2024-04-22T06:31:07.027113+0000","last_became_peered":"2024-04-22T06:31:07.027113+0000","last_unstale":"2024-04-22T06:31:12.031619+0000","last_undegraded":"2024-04-22T06:31:12.031619+0000","last_fullsized":"2024-04-22T06:31:12.031619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:08:42.204674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559261+0000","last_change":"2024-04-22T06:31:07.015520+0000","last_active":"2024-04-22T06:31:12.559261+0000","last_peered":"2024-04-22T06:31:12.559261+0000","last_clean":"2024-04-22T06:31:12.559261+0000","last_became_active":"2024-04-22T06:31:07.015199+0000","last_became_peered":"2024-04-22T06:31:07.015199+0000","last_unstale":"2024-04-22T06:31:12.559261+0000","last_undegraded":"2024-04-22T06:31:12.559261+0000","last_fullsized":"2024-04-22T06:31:12.559261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:48:40.688931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031281+0000","last_change":"2024-04-22T06:31:07.016457+0000","last_active":"2024-04-22T06:31:12.031281+0000","last_peered":"2024-04-22T06:31:12.031281+0000","last_clean":"2024-04-22T06:31:12.031281+0000","last_became_active":"2024-04-22T06:31:07.016082+0000","last_became_peered":"2024-04-22T06:31:07.016082+0000","last_unstale":"2024-04-22T06:31:12.031281+0000","last_undegraded":"2024-04-22T06:31:12.031281+0000","last_fullsized":"2024-04-22T06:31:12.031281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:12:28.517065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030497+0000","last_change":"2024-04-22T06:31:07.019955+0000","last_active":"2024-04-22T06:31:12.030497+0000","last_peered":"2024-04-22T06:31:12.030497+0000","last_clean":"2024-04-22T06:31:12.030497+0000","last_became_active":"2024-04-22T06:31:07.019174+0000","last_became_peered":"2024-04-22T06:31:07.019174+0000","last_unstale":"2024-04-22T06:31:12.030497+0000","last_undegraded":"2024-04-22T06:31:12.030497+0000","last_fullsized":"2024-04-22T06:31:12.030497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:09:07.106161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560474+0000","last_change":"2024-04-22T06:31:07.019341+0000","last_active":"2024-04-22T06:31:12.560474+0000","last_peered":"2024-04-22T06:31:12.560474+0000","last_clean":"2024-04-22T06:31:12.560474+0000","last_became_active":"2024-04-22T06:31:07.019048+0000","last_became_peered":"2024-04-22T06:31:07.019048+0000","last_unstale":"2024-04-22T06:31:12.560474+0000","last_undegraded":"2024-04-22T06:31:12.560474+0000","last_fullsized":"2024-04-22T06:31:12.560474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:04:19.362686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559337+0000","last_change":"2024-04-22T06:31:07.019433+0000","last_active":"2024-04-22T06:31:12.559337+0000","last_peered":"2024-04-22T06:31:12.559337+0000","last_clean":"2024-04-22T06:31:12.559337+0000","last_became_active":"2024-04-22T06:31:07.019133+0000","last_became_peered":"2024-04-22T06:31:07.019133+0000","last_unstale":"2024-04-22T06:31:12.559337+0000","last_undegraded":"2024-04-22T06:31:12.559337+0000","last_fullsized":"2024-04-22T06:31:12.559337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:01:54.466486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.561087+0000","last_change":"2024-04-22T06:31:07.023014+0000","last_active":"2024-04-22T06:31:12.561087+0000","last_peered":"2024-04-22T06:31:12.561087+0000","last_clean":"2024-04-22T06:31:12.561087+0000","last_became_active":"2024-04-22T06:31:07.022716+0000","last_became_peered":"2024-04-22T06:31:07.022716+0000","last_unstale":"2024-04-22T06:31:12.561087+0000","last_undegraded":"2024-04-22T06:31:12.561087+0000","last_fullsized":"2024-04-22T06:31:12.561087+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:11:47.309706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.033363+0000","last_change":"2024-04-22T06:31:07.020446+0000","last_active":"2024-04-22T06:31:12.033363+0000","last_peered":"2024-04-22T06:31:12.033363+0000","last_clean":"2024-04-22T06:31:12.033363+0000","last_became_active":"2024-04-22T06:31:07.018995+0000","last_became_peered":"2024-04-22T06:31:07.018995+0000","last_unstale":"2024-04-22T06:31:12.033363+0000","last_undegraded":"2024-04-22T06:31:12.033363+0000","last_fullsized":"2024-04-22T06:31:12.033363+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:56:45.031262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.558848+0000","last_change":"2024-04-22T06:31:07.014927+0000","last_active":"2024-04-22T06:31:12.558848+0000","last_peered":"2024-04-22T06:31:12.558848+0000","last_clean":"2024-04-22T06:31:12.558848+0000","last_became_active":"2024-04-22T06:31:07.014652+0000","last_became_peered":"2024-04-22T06:31:07.014652+0000","last_unstale":"2024-04-22T06:31:12.558848+0000","last_undegraded":"2024-04-22T06:31:12.558848+0000","last_fullsized":"2024-04-22T06:31:12.558848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:51:33.506667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559161+0000","last_change":"2024-04-22T06:31:07.014209+0000","last_active":"2024-04-22T06:31:12.559161+0000","last_peered":"2024-04-22T06:31:12.559161+0000","last_clean":"2024-04-22T06:31:12.559161+0000","last_became_active":"2024-04-22T06:31:07.013946+0000","last_became_peered":"2024-04-22T06:31:07.013946+0000","last_unstale":"2024-04-22T06:31:12.559161+0000","last_undegraded":"2024-04-22T06:31:12.559161+0000","last_fullsized":"2024-04-22T06:31:12.559161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:40:31.669158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031004+0000","last_change":"2024-04-22T06:31:07.008764+0000","last_active":"2024-04-22T06:31:12.031004+0000","last_peered":"2024-04-22T06:31:12.031004+0000","last_clean":"2024-04-22T06:31:12.031004+0000","last_became_active":"2024-04-22T06:31:07.007810+0000","last_became_peered":"2024-04-22T06:31:07.007810+0000","last_unstale":"2024-04-22T06:31:12.031004+0000","last_undegraded":"2024-04-22T06:31:12.031004+0000","last_fullsized":"2024-04-22T06:31:12.031004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:49:54.894818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.559408+0000","last_change":"2024-04-22T06:31:07.022141+0000","last_active":"2024-04-22T06:31:12.559408+0000","last_peered":"2024-04-22T06:31:12.559408+0000","last_clean":"2024-04-22T06:31:12.559408+0000","last_became_active":"2024-04-22T06:31:07.021845+0000","last_became_peered":"2024-04-22T06:31:07.021845+0000","last_unstale":"2024-04-22T06:31:12.559408+0000","last_undegraded":"2024-04-22T06:31:12.559408+0000","last_fullsized":"2024-04-22T06:31:12.559408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:26:28.598104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.029984+0000","last_change":"2024-04-22T06:31:07.012188+0000","last_active":"2024-04-22T06:31:12.029984+0000","last_peered":"2024-04-22T06:31:12.029984+0000","last_clean":"2024-04-22T06:31:12.029984+0000","last_became_active":"2024-04-22T06:31:07.011909+0000","last_became_peered":"2024-04-22T06:31:07.011909+0000","last_unstale":"2024-04-22T06:31:12.029984+0000","last_undegraded":"2024-04-22T06:31:12.029984+0000","last_fullsized":"2024-04-22T06:31:12.029984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:25:13.089928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031293+0000","last_change":"2024-04-22T06:31:07.020033+0000","last_active":"2024-04-22T06:31:12.031293+0000","last_peered":"2024-04-22T06:31:12.031293+0000","last_clean":"2024-04-22T06:31:12.031293+0000","last_became_active":"2024-04-22T06:31:07.019689+0000","last_became_peered":"2024-04-22T06:31:07.019689+0000","last_unstale":"2024-04-22T06:31:12.031293+0000","last_undegraded":"2024-04-22T06:31:12.031293+0000","last_fullsized":"2024-04-22T06:31:12.031293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:09:07.209203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030666+0000","last_change":"2024-04-22T06:31:07.027287+0000","last_active":"2024-04-22T06:31:12.030666+0000","last_peered":"2024-04-22T06:31:12.030666+0000","last_clean":"2024-04-22T06:31:12.030666+0000","last_became_active":"2024-04-22T06:31:07.027015+0000","last_became_peered":"2024-04-22T06:31:07.027015+0000","last_unstale":"2024-04-22T06:31:12.030666+0000","last_undegraded":"2024-04-22T06:31:12.030666+0000","last_fullsized":"2024-04-22T06:31:12.030666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:42:49.322413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.031959+0000","last_change":"2024-04-22T06:31:07.026332+0000","last_active":"2024-04-22T06:31:12.031959+0000","last_peered":"2024-04-22T06:31:12.031959+0000","last_clean":"2024-04-22T06:31:12.031959+0000","last_became_active":"2024-04-22T06:31:07.026018+0000","last_became_peered":"2024-04-22T06:31:07.026018+0000","last_unstale":"2024-04-22T06:31:12.031959+0000","last_undegraded":"2024-04-22T06:31:12.031959+0000","last_fullsized":"2024-04-22T06:31:12.031959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:27:08.560056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560402+0000","last_change":"2024-04-22T06:31:07.020497+0000","last_active":"2024-04-22T06:31:12.560402+0000","last_peered":"2024-04-22T06:31:12.560402+0000","last_clean":"2024-04-22T06:31:12.560402+0000","last_became_active":"2024-04-22T06:31:07.020194+0000","last_became_peered":"2024-04-22T06:31:07.020194+0000","last_unstale":"2024-04-22T06:31:12.560402+0000","last_undegraded":"2024-04-22T06:31:12.560402+0000","last_fullsized":"2024-04-22T06:31:12.560402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:56:59.765295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.560664+0000","last_change":"2024-04-22T06:31:07.018396+0000","last_active":"2024-04-22T06:31:12.560664+0000","last_peered":"2024-04-22T06:31:12.560664+0000","last_clean":"2024-04-22T06:31:12.560664+0000","last_became_active":"2024-04-22T06:31:07.018073+0000","last_became_peered":"2024-04-22T06:31:07.018073+0000","last_unstale":"2024-04-22T06:31:12.560664+0000","last_undegraded":"2024-04-22T06:31:12.560664+0000","last_fullsized":"2024-04-22T06:31:12.560664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:37:06.501499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-22T06:31:12.030917+0000","last_change":"2024-04-22T06:31:07.014439+0000","last_active":"2024-04-22T06:31:12.030917+0000","last_peered":"2024-04-22T06:31:12.030917+0000","last_clean":"2024-04-22T06:31:12.030917+0000","last_became_active":"2024-04-22T06:31:07.014124+0000","last_became_peered":"2024-04-22T06:31:07.014124+0000","last_unstale":"2024-04-22T06:31:12.030917+0000","last_undegraded":"2024-04-22T06:31:12.030917+0000","last_fullsized":"2024-04-22T06:31:12.030917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:58:46.075068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6046,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":12092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":880640,"data_stored":696611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":282624,"data_stored":108859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":278528,"data_stored":105689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":278528,"data_stored":105053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":286720,"data_stored":109391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574860,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":105759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":876544,"data_stored":695955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T06:31:41.133 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-22T06:31:41.133 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-22T06:31:41.133 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-22T06:31:41.133 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-22T06:31:41.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.269+0000 7f6081093640 1 Processor -- start 2024-04-22T06:31:41.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.269+0000 7f6081093640 1 -- start start 2024-04-22T06:31:41.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.270+0000 7f6081093640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f607c155460 0x7f607c15f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:41.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.270+0000 7f6081093640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f607c0cc6d0 0x7f607c073230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:41.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.270+0000 7f6081093640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f607c0cd0a0 con 0x7f607c0ccb40 2024-04-22T06:31:41.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.270+0000 7f6081093640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f607c155840 con 0x7f607c0cc6d0 2024-04-22T06:31:41.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.270+0000 7f6081093640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f607c0d1520 con 0x7f607c155460 2024-04-22T06:31:41.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.270+0000 7f607a59c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f607c0cc6d0 0x7f607c073230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:41.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.270+0000 7f607b59e640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f607c0ccb40 0x7f607c0c5c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:49552/0 (socket says 172.21.15.44:49552) 2024-04-22T06:31:41.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.270+0000 7f607b59e640 1 -- 172.21.15.44:0/4071202288 learned_addr learned my addr 172.21.15.44:0/4071202288 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:41.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.270+0000 7f607ad9d640 1 --2- 172.21.15.44:0/4071202288 >> v2:172.21.15.70:3300/0 conn(0x7f607c155460 0x7f607c15f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:41.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.270+0000 7f607a59c640 1 -- 172.21.15.44:0/4071202288 >> v2:172.21.15.70:3300/0 conn(0x7f607c155460 msgr2=0x7f607c15f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:41.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.270+0000 7f607a59c640 1 --2- 172.21.15.44:0/4071202288 >> v2:172.21.15.70:3300/0 conn(0x7f607c155460 0x7f607c15f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:41.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.270+0000 7f607a59c640 1 -- 172.21.15.44:0/4071202288 >> v1:172.21.15.44:6789/0 conn(0x7f607c0ccb40 legacy=0x7f607c0c5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:41.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.270+0000 7f607a59c640 1 -- 172.21.15.44:0/4071202288 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f607c0d0910 con 0x7f607c0cc6d0 2024-04-22T06:31:41.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.271+0000 7f607a59c640 1 --2- 172.21.15.44:0/4071202288 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f607c0cc6d0 0x7f607c073230 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f607c0cd380 tx=0x7f607002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb25fc5f09225d9e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:41.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.271+0000 7f6079d9b640 1 -- 172.21.15.44:0/4071202288 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6070035020 con 0x7f607c0cc6d0 2024-04-22T06:31:41.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.271+0000 7f6079d9b640 1 -- 172.21.15.44:0/4071202288 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6070031830 con 0x7f607c0cc6d0 2024-04-22T06:31:41.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.271+0000 7f6079d9b640 1 -- 172.21.15.44:0/4071202288 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6070031c00 con 0x7f607c0cc6d0 2024-04-22T06:31:41.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.272+0000 7f6081093640 1 -- 172.21.15.44:0/4071202288 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f607c0cc6d0 msgr2=0x7f607c073230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:41.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.272+0000 7f6081093640 1 --2- 172.21.15.44:0/4071202288 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f607c0cc6d0 0x7f607c073230 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f607c0cd380 tx=0x7f607002b8d0 comp rx=0 tx=0).stop 2024-04-22T06:31:41.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.272+0000 7f6081093640 1 -- 172.21.15.44:0/4071202288 shutdown_connections 2024-04-22T06:31:41.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.272+0000 7f6081093640 1 --2- 172.21.15.44:0/4071202288 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f607c0cc6d0 0x7f607c073230 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:41.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.272+0000 7f6081093640 1 --2- 172.21.15.44:0/4071202288 >> v2:172.21.15.70:3300/0 conn(0x7f607c155460 0x7f607c15f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:41.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.272+0000 7f6081093640 1 -- 172.21.15.44:0/4071202288 >> 172.21.15.44:0/4071202288 conn(0x7f607c06cca0 msgr2=0x7f607c06d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:41.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.272+0000 7f6081093640 1 -- 172.21.15.44:0/4071202288 shutdown_connections 2024-04-22T06:31:41.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.272+0000 7f6081093640 1 -- 172.21.15.44:0/4071202288 wait complete. 2024-04-22T06:31:41.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.273+0000 7f6081093640 1 Processor -- start 2024-04-22T06:31:41.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.273+0000 7f6081093640 1 -- start start 2024-04-22T06:31:41.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.274+0000 7f6081093640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f607c0cc6d0 0x7f607c0cffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:41.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.274+0000 7f6081093640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f607c0ccb40 0x7f607c152080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:41.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.274+0000 7f6081093640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f607c155460 0x7f607c1525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:41.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.274+0000 7f6081093640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f607c0d11b0 con 0x7f607c155460 2024-04-22T06:31:41.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.274+0000 7f6081093640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f607c0d0ed0 con 0x7f607c0cc6d0 2024-04-22T06:31:41.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.274+0000 7f6081093640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f607c0c6310 con 0x7f607c0ccb40 2024-04-22T06:31:41.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.274+0000 7f607b59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f607c155460 0x7f607c1525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:41.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.274+0000 7f607ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f607c0cc6d0 0x7f607c0cffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:41.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.274+0000 7f607b59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f607c155460 0x7f607c1525c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53952/0 (socket says 172.21.15.44:53952) 2024-04-22T06:31:41.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.274+0000 7f607b59e640 1 -- 172.21.15.44:0/550097705 learned_addr learned my addr 172.21.15.44:0/550097705 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:41.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.274+0000 7f607a59c640 1 --2- 172.21.15.44:0/550097705 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f607c0ccb40 0x7f607c152080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:41.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.274+0000 7f607ad9d640 1 -- 172.21.15.44:0/550097705 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f607c0ccb40 msgr2=0x7f607c152080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:41.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.274+0000 7f607ad9d640 1 --2- 172.21.15.44:0/550097705 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f607c0ccb40 0x7f607c152080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:41.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.274+0000 7f607ad9d640 1 -- 172.21.15.44:0/550097705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f607c155460 msgr2=0x7f607c1525c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:41.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.274+0000 7f607ad9d640 1 --2- 172.21.15.44:0/550097705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f607c155460 0x7f607c1525c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:41.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.274+0000 7f607ad9d640 1 -- 172.21.15.44:0/550097705 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f607c14a1b0 con 0x7f607c0cc6d0 2024-04-22T06:31:41.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.274+0000 7f607b59e640 1 --2- 172.21.15.44:0/550097705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f607c155460 0x7f607c1525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:31:41.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.275+0000 7f607ad9d640 1 --2- 172.21.15.44:0/550097705 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f607c0cc6d0 0x7f607c0cffe0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f607400e870 tx=0x7f607400ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:41.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.275+0000 7f60637fe640 1 -- 172.21.15.44:0/550097705 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6074017070 con 0x7f607c0cc6d0 2024-04-22T06:31:41.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.275+0000 7f60637fe640 1 -- 172.21.15.44:0/550097705 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f607400ee60 con 0x7f607c0cc6d0 2024-04-22T06:31:41.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.275+0000 7f6081093640 1 -- 172.21.15.44:0/550097705 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f607c0c87a0 con 0x7f607c0cc6d0 2024-04-22T06:31:41.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.275+0000 7f60637fe640 1 -- 172.21.15.44:0/550097705 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607400cb60 con 0x7f607c0cc6d0 2024-04-22T06:31:41.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.275+0000 7f6081093640 1 -- 172.21.15.44:0/550097705 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f607c0c8d60 con 0x7f607c0cc6d0 2024-04-22T06:31:41.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.276+0000 7f6081093640 1 -- 172.21.15.44:0/550097705 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f607c068cc0 con 0x7f607c0cc6d0 2024-04-22T06:31:41.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.276+0000 7f60637fe640 1 -- 172.21.15.44:0/550097705 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f607401a070 con 0x7f607c0cc6d0 2024-04-22T06:31:41.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.276+0000 7f60637fe640 1 --2- 172.21.15.44:0/550097705 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f606806c920 0x7f606806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:41.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.277+0000 7f60637fe640 1 -- 172.21.15.44:0/550097705 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f607408bc80 con 0x7f607c0cc6d0 2024-04-22T06:31:41.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.277+0000 7f607a59c640 1 --2- 172.21.15.44:0/550097705 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f606806c920 0x7f606806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:41.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.277+0000 7f607a59c640 1 --2- 172.21.15.44:0/550097705 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f606806c920 0x7f606806ede0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6070036000 tx=0x7f6070008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:41.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.281+0000 7f60637fe640 1 -- 172.21.15.44:0/550097705 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6074059690 con 0x7f607c0cc6d0 2024-04-22T06:31:41.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.503+0000 7f6081093640 1 -- 172.21.15.44:0/550097705 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f607c0cb650 con 0x7f607c0cc6d0 2024-04-22T06:31:41.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.504+0000 7f60637fe640 1 -- 172.21.15.44:0/550097705 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7f607405d990 con 0x7f607c0cc6d0 2024-04-22T06:31:41.502 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:31:41.503 INFO:teuthology.orchestra.run.smithi044.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-04-22T06:31:41.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.506+0000 7f6081093640 1 -- 172.21.15.44:0/550097705 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f606806c920 msgr2=0x7f606806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:41.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.506+0000 7f6081093640 1 --2- 172.21.15.44:0/550097705 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f606806c920 0x7f606806ede0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6070036000 tx=0x7f6070008030 comp rx=0 tx=0).stop 2024-04-22T06:31:41.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.506+0000 7f6081093640 1 -- 172.21.15.44:0/550097705 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f607c0cc6d0 msgr2=0x7f607c0cffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:41.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.506+0000 7f6081093640 1 --2- 172.21.15.44:0/550097705 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f607c0cc6d0 0x7f607c0cffe0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f607400e870 tx=0x7f607400ed30 comp rx=0 tx=0).stop 2024-04-22T06:31:41.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.506+0000 7f6081093640 1 -- 172.21.15.44:0/550097705 shutdown_connections 2024-04-22T06:31:41.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.506+0000 7f6081093640 1 --2- 172.21.15.44:0/550097705 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f606806c920 0x7f606806ede0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:41.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.506+0000 7f6081093640 1 --2- 172.21.15.44:0/550097705 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f607c0ccb40 0x7f607c152080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:41.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.506+0000 7f6081093640 1 --2- 172.21.15.44:0/550097705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f607c155460 0x7f607c1525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:41.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.506+0000 7f6081093640 1 --2- 172.21.15.44:0/550097705 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f607c0cc6d0 0x7f607c0cffe0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:41.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.506+0000 7f6081093640 1 -- 172.21.15.44:0/550097705 >> 172.21.15.44:0/550097705 conn(0x7f607c06cca0 msgr2=0x7f607c073230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:41.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.506+0000 7f6081093640 1 -- 172.21.15.44:0/550097705 shutdown_connections 2024-04-22T06:31:41.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:41.506+0000 7f6081093640 1 -- 172.21.15.44:0/550097705 wait complete. 2024-04-22T06:31:44.516 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-22T06:31:44.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.630+0000 7f6d95915640 1 Processor -- start 2024-04-22T06:31:44.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.630+0000 7f6d95915640 1 -- start start 2024-04-22T06:31:44.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.631+0000 7f6d95915640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d90155010 0x7f6d9015bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:44.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.631+0000 7f6d95915640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d900d0fa0 con 0x7f6d9006d5a0 2024-04-22T06:31:44.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.631+0000 7f6d95915640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d900d1280 con 0x7f6d901553f0 2024-04-22T06:31:44.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.631+0000 7f6d95915640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f6d900d1560 con 0x7f6d90155010 2024-04-22T06:31:44.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.631+0000 7f6d8f7fe640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f6d9006d5a0 0x7f6d900cbbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:32866/0 (socket says 172.21.15.44:32866) 2024-04-22T06:31:44.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.631+0000 7f6d8f7fe640 1 -- 172.21.15.44:0/71376606 learned_addr learned my addr 172.21.15.44:0/71376606 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:44.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.631+0000 7f6d8effd640 1 --2- 172.21.15.44:0/71376606 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d90155010 0x7f6d9015bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:44.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.632+0000 7f6d8dffb640 1 -- 172.21.15.44:0/71376606 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3375082722 0 0) 0x7f6d900d0fa0 con 0x7f6d9006d5a0 2024-04-22T06:31:44.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.632+0000 7f6d8effd640 1 -- 172.21.15.44:0/71376606 >> v1:172.21.15.70:6789/0 conn(0x7f6d901553f0 legacy=0x7f6d900c6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:44.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.632+0000 7f6d8effd640 1 -- 172.21.15.44:0/71376606 >> v1:172.21.15.44:6789/0 conn(0x7f6d9006d5a0 legacy=0x7f6d900cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:44.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.632+0000 7f6d8effd640 1 -- 172.21.15.44:0/71376606 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d900c5d50 con 0x7f6d90155010 2024-04-22T06:31:44.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.633+0000 7f6d8effd640 1 --2- 172.21.15.44:0/71376606 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d90155010 0x7f6d9015bc00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6d88009dc0 tx=0x7f6d8802be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3825efc824aba65a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:44.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.633+0000 7f6d8dffb640 1 -- 172.21.15.44:0/71376606 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d88039070 con 0x7f6d90155010 2024-04-22T06:31:44.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.633+0000 7f6d8dffb640 1 -- 172.21.15.44:0/71376606 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d8802cb30 con 0x7f6d90155010 2024-04-22T06:31:44.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.633+0000 7f6d8dffb640 1 -- 172.21.15.44:0/71376606 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d88034440 con 0x7f6d90155010 2024-04-22T06:31:44.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.633+0000 7f6d95915640 1 -- 172.21.15.44:0/71376606 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d90155010 msgr2=0x7f6d9015bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:44.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.633+0000 7f6d95915640 1 --2- 172.21.15.44:0/71376606 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d90155010 0x7f6d9015bc00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6d88009dc0 tx=0x7f6d8802be00 comp rx=0 tx=0).stop 2024-04-22T06:31:44.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.633+0000 7f6d95915640 1 -- 172.21.15.44:0/71376606 shutdown_connections 2024-04-22T06:31:44.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.633+0000 7f6d95915640 1 --2- 172.21.15.44:0/71376606 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d90155010 0x7f6d9015bc00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:44.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.633+0000 7f6d95915640 1 -- 172.21.15.44:0/71376606 >> 172.21.15.44:0/71376606 conn(0x7f6d900750d0 msgr2=0x7f6d900754e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:44.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.633+0000 7f6d95915640 1 -- 172.21.15.44:0/71376606 shutdown_connections 2024-04-22T06:31:44.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.633+0000 7f6d95915640 1 -- 172.21.15.44:0/71376606 wait complete. 2024-04-22T06:31:44.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.635+0000 7f6d95915640 1 Processor -- start 2024-04-22T06:31:44.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.635+0000 7f6d95915640 1 -- start start 2024-04-22T06:31:44.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.635+0000 7f6d95915640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d9006d5a0 0x7f6d900cff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:44.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.635+0000 7f6d95915640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d90155010 0x7f6d901f1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:44.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.635+0000 7f6d95915640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d901553f0 0x7f6d901f4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:44.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.635+0000 7f6d95915640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6d900cb450 con 0x7f6d9006d5a0 2024-04-22T06:31:44.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.635+0000 7f6d95915640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6d900cb2d0 con 0x7f6d901553f0 2024-04-22T06:31:44.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.635+0000 7f6d95915640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f6d900cc6f0 con 0x7f6d90155010 2024-04-22T06:31:44.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.636+0000 7f6d8effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d9006d5a0 0x7f6d900cff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:44.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.636+0000 7f6d8effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d9006d5a0 0x7f6d900cff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48612/0 (socket says 172.21.15.44:48612) 2024-04-22T06:31:44.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.636+0000 7f6d8effd640 1 -- 172.21.15.44:0/2724088969 learned_addr learned my addr 172.21.15.44:0/2724088969 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:44.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.636+0000 7f6d8f7fe640 1 --2- 172.21.15.44:0/2724088969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d901553f0 0x7f6d901f4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:44.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.636+0000 7f6d8e7fc640 1 --2- 172.21.15.44:0/2724088969 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d90155010 0x7f6d901f1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:44.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.636+0000 7f6d8effd640 1 -- 172.21.15.44:0/2724088969 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d90155010 msgr2=0x7f6d901f1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:44.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.636+0000 7f6d8effd640 1 --2- 172.21.15.44:0/2724088969 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d90155010 0x7f6d901f1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:44.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.636+0000 7f6d8effd640 1 -- 172.21.15.44:0/2724088969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d901553f0 msgr2=0x7f6d901f4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:44.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.636+0000 7f6d8effd640 1 --2- 172.21.15.44:0/2724088969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d901553f0 0x7f6d901f4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:44.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.636+0000 7f6d8effd640 1 -- 172.21.15.44:0/2724088969 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d90159200 con 0x7f6d9006d5a0 2024-04-22T06:31:44.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.636+0000 7f6d8effd640 1 --2- 172.21.15.44:0/2724088969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d9006d5a0 0x7f6d900cff70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6d88009dc0 tx=0x7f6d88004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:44.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.636+0000 7f6d87fff640 1 -- 172.21.15.44:0/2724088969 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d88044020 con 0x7f6d9006d5a0 2024-04-22T06:31:44.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.636+0000 7f6d87fff640 1 -- 172.21.15.44:0/2724088969 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d8803f670 con 0x7f6d9006d5a0 2024-04-22T06:31:44.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.636+0000 7f6d95915640 1 -- 172.21.15.44:0/2724088969 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d901594c0 con 0x7f6d9006d5a0 2024-04-22T06:31:44.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.636+0000 7f6d87fff640 1 -- 172.21.15.44:0/2724088969 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d88039040 con 0x7f6d9006d5a0 2024-04-22T06:31:44.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.636+0000 7f6d95915640 1 -- 172.21.15.44:0/2724088969 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d90159a80 con 0x7f6d9006d5a0 2024-04-22T06:31:44.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.637+0000 7f6d95915640 1 -- 172.21.15.44:0/2724088969 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d900c6e90 con 0x7f6d9006d5a0 2024-04-22T06:31:44.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.638+0000 7f6d87fff640 1 -- 172.21.15.44:0/2724088969 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6d88047020 con 0x7f6d9006d5a0 2024-04-22T06:31:44.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.638+0000 7f6d87fff640 1 --2- 172.21.15.44:0/2724088969 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f6d7006c920 0x7f6d7006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:44.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.638+0000 7f6d87fff640 1 -- 172.21.15.44:0/2724088969 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6d880ae3c0 con 0x7f6d9006d5a0 2024-04-22T06:31:44.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.639+0000 7f6d8e7fc640 1 --2- 172.21.15.44:0/2724088969 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f6d7006c920 0x7f6d7006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:44.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.639+0000 7f6d8e7fc640 1 --2- 172.21.15.44:0/2724088969 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f6d7006c920 0x7f6d7006ede0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6d901538c0 tx=0x7f6d78009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:44.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.643+0000 7f6d87fff640 1 -- 172.21.15.44:0/2724088969 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6d900c6e90 con 0x7f6d9006d5a0 2024-04-22T06:31:44.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.867+0000 7f6d95915640 1 -- 172.21.15.44:0/2724088969 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f6d901f1110 con 0x7f6d9006d5a0 2024-04-22T06:31:44.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.868+0000 7f6d87fff640 1 -- 172.21.15.44:0/2724088969 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7f6d8804f020 con 0x7f6d9006d5a0 2024-04-22T06:31:44.867 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:31:44.867 INFO:teuthology.orchestra.run.smithi044.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-04-22T06:31:44.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.871+0000 7f6d95915640 1 -- 172.21.15.44:0/2724088969 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f6d7006c920 msgr2=0x7f6d7006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:44.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.871+0000 7f6d95915640 1 --2- 172.21.15.44:0/2724088969 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f6d7006c920 0x7f6d7006ede0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6d901538c0 tx=0x7f6d78009040 comp rx=0 tx=0).stop 2024-04-22T06:31:44.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.871+0000 7f6d95915640 1 -- 172.21.15.44:0/2724088969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d9006d5a0 msgr2=0x7f6d900cff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:44.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.871+0000 7f6d95915640 1 --2- 172.21.15.44:0/2724088969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d9006d5a0 0x7f6d900cff70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6d88009dc0 tx=0x7f6d88004a40 comp rx=0 tx=0).stop 2024-04-22T06:31:44.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.871+0000 7f6d95915640 1 -- 172.21.15.44:0/2724088969 shutdown_connections 2024-04-22T06:31:44.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.871+0000 7f6d95915640 1 --2- 172.21.15.44:0/2724088969 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f6d7006c920 0x7f6d7006ede0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:44.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.871+0000 7f6d95915640 1 --2- 172.21.15.44:0/2724088969 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d90155010 0x7f6d901f1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:44.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.871+0000 7f6d95915640 1 --2- 172.21.15.44:0/2724088969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d901553f0 0x7f6d901f4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:44.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.871+0000 7f6d95915640 1 --2- 172.21.15.44:0/2724088969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d9006d5a0 0x7f6d900cff70 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:44.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.871+0000 7f6d95915640 1 -- 172.21.15.44:0/2724088969 >> 172.21.15.44:0/2724088969 conn(0x7f6d900750d0 msgr2=0x7f6d900cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:44.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.871+0000 7f6d95915640 1 -- 172.21.15.44:0/2724088969 shutdown_connections 2024-04-22T06:31:44.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:44.871+0000 7f6d95915640 1 -- 172.21.15.44:0/2724088969 wait complete. 2024-04-22T06:31:47.886 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-22T06:31:47.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.001+0000 7f9112c5c640 1 Processor -- start 2024-04-22T06:31:48.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.001+0000 7f9112c5c640 1 -- start start 2024-04-22T06:31:48.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.001+0000 7f9112c5c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f910c151f50 0x7f910c158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:48.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.002+0000 7f9112c5c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f910c0c8410 0x7f910c0cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:48.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.002+0000 7f9112c5c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f910c158db0 con 0x7f910c0cc980 2024-04-22T06:31:48.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.002+0000 7f9112c5c640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f910c15b770 con 0x7f910c0c8410 2024-04-22T06:31:48.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.002+0000 7f9112c5c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f910c15ba50 con 0x7f910c151f50 2024-04-22T06:31:48.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.002+0000 7f910bfff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f910c0c8410 0x7f910c0cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:48.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.002+0000 7f910bfff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f910c0c8410 0x7f910c0cc440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48628/0 (socket says 172.21.15.44:48628) 2024-04-22T06:31:48.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.002+0000 7f910bfff640 1 -- 172.21.15.44:0/385932103 learned_addr learned my addr 172.21.15.44:0/385932103 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:48.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.002+0000 7f91109f8640 1 --2- 172.21.15.44:0/385932103 >> v2:172.21.15.70:3300/0 conn(0x7f910c151f50 0x7f910c158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:48.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.002+0000 7f910bfff640 1 -- 172.21.15.44:0/385932103 >> v2:172.21.15.70:3300/0 conn(0x7f910c151f50 msgr2=0x7f910c158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:48.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.002+0000 7f910bfff640 1 --2- 172.21.15.44:0/385932103 >> v2:172.21.15.70:3300/0 conn(0x7f910c151f50 0x7f910c158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:48.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.002+0000 7f910bfff640 1 -- 172.21.15.44:0/385932103 >> v1:172.21.15.70:6789/0 conn(0x7f910c0cc980 legacy=0x7f910c1598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:48.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.002+0000 7f910bfff640 1 -- 172.21.15.44:0/385932103 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f910c15bd30 con 0x7f910c0c8410 2024-04-22T06:31:48.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.003+0000 7f910bfff640 1 --2- 172.21.15.44:0/385932103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f910c0c8410 0x7f910c0cc440 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f910c0c8130 tx=0x7f90f402ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=555698ef7b4c21dd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:48.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.003+0000 7f910b7fe640 1 -- 172.21.15.44:0/385932103 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f402f850 con 0x7f910c0c8410 2024-04-22T06:31:48.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.003+0000 7f910b7fe640 1 -- 172.21.15.44:0/385932103 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90f402f9f0 con 0x7f910c0c8410 2024-04-22T06:31:48.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.003+0000 7f910b7fe640 1 -- 172.21.15.44:0/385932103 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f402fdc0 con 0x7f910c0c8410 2024-04-22T06:31:48.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.004+0000 7f9112c5c640 1 -- 172.21.15.44:0/385932103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f910c0c8410 msgr2=0x7f910c0cc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:48.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.004+0000 7f9112c5c640 1 --2- 172.21.15.44:0/385932103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f910c0c8410 0x7f910c0cc440 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f910c0c8130 tx=0x7f90f402ba30 comp rx=0 tx=0).stop 2024-04-22T06:31:48.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.004+0000 7f9112c5c640 1 -- 172.21.15.44:0/385932103 shutdown_connections 2024-04-22T06:31:48.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.004+0000 7f9112c5c640 1 --2- 172.21.15.44:0/385932103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f910c0c8410 0x7f910c0cc440 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:48.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.004+0000 7f9112c5c640 1 --2- 172.21.15.44:0/385932103 >> v2:172.21.15.70:3300/0 conn(0x7f910c151f50 0x7f910c158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:48.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.004+0000 7f9112c5c640 1 -- 172.21.15.44:0/385932103 >> 172.21.15.44:0/385932103 conn(0x7f910c06d530 msgr2=0x7f910c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:48.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.004+0000 7f9112c5c640 1 -- 172.21.15.44:0/385932103 shutdown_connections 2024-04-22T06:31:48.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.004+0000 7f9112c5c640 1 -- 172.21.15.44:0/385932103 wait complete. 2024-04-22T06:31:48.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.005+0000 7f9112c5c640 1 Processor -- start 2024-04-22T06:31:48.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.005+0000 7f9112c5c640 1 -- start start 2024-04-22T06:31:48.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.006+0000 7f9112c5c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f910c0c8410 0x7f910c0c6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:48.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.006+0000 7f9112c5c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f910c0cc980 0x7f910c0724d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:48.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.006+0000 7f9112c5c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f910c151f50 0x7f910c0733b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:48.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.006+0000 7f9112c5c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f910c159380 con 0x7f910c151f50 2024-04-22T06:31:48.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.006+0000 7f9112c5c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f910c15c2f0 con 0x7f910c0cc980 2024-04-22T06:31:48.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.006+0000 7f9112c5c640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f910c15b3e0 con 0x7f910c0c8410 2024-04-22T06:31:48.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.006+0000 7f91111f9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f910c151f50 0x7f910c0733b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:48.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.006+0000 7f91111f9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f910c151f50 0x7f910c0733b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48638/0 (socket says 172.21.15.44:48638) 2024-04-22T06:31:48.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.006+0000 7f91111f9640 1 -- 172.21.15.44:0/1119936576 learned_addr learned my addr 172.21.15.44:0/1119936576 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:48.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.006+0000 7f910bfff640 1 --2- 172.21.15.44:0/1119936576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f910c0cc980 0x7f910c0724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:48.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.006+0000 7f91109f8640 1 --2- 172.21.15.44:0/1119936576 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f910c0c8410 0x7f910c0c6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:48.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.006+0000 7f91111f9640 1 -- 172.21.15.44:0/1119936576 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f910c0c8410 msgr2=0x7f910c0c6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:48.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.006+0000 7f91111f9640 1 --2- 172.21.15.44:0/1119936576 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f910c0c8410 0x7f910c0c6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:48.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.006+0000 7f91111f9640 1 -- 172.21.15.44:0/1119936576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f910c0cc980 msgr2=0x7f910c0724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:48.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.006+0000 7f91111f9640 1 --2- 172.21.15.44:0/1119936576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f910c0cc980 0x7f910c0724d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:48.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.006+0000 7f91111f9640 1 -- 172.21.15.44:0/1119936576 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f910c152d10 con 0x7f910c151f50 2024-04-22T06:31:48.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.007+0000 7f91111f9640 1 --2- 172.21.15.44:0/1119936576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f910c151f50 0x7f910c0733b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f90fc00c400 tx=0x7f90fc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:48.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.007+0000 7f91097fa640 1 -- 172.21.15.44:0/1119936576 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90fc017070 con 0x7f910c151f50 2024-04-22T06:31:48.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.007+0000 7f91097fa640 1 -- 172.21.15.44:0/1119936576 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90fc0052d0 con 0x7f910c151f50 2024-04-22T06:31:48.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.007+0000 7f91097fa640 1 -- 172.21.15.44:0/1119936576 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90fc003810 con 0x7f910c151f50 2024-04-22T06:31:48.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.007+0000 7f9112c5c640 1 -- 172.21.15.44:0/1119936576 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f910c1f1770 con 0x7f910c151f50 2024-04-22T06:31:48.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.007+0000 7f9112c5c640 1 -- 172.21.15.44:0/1119936576 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f910c0cb740 con 0x7f910c151f50 2024-04-22T06:31:48.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.008+0000 7f9112c5c640 1 -- 172.21.15.44:0/1119936576 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f910c068cc0 con 0x7f910c151f50 2024-04-22T06:31:48.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.008+0000 7f91097fa640 1 -- 172.21.15.44:0/1119936576 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f90fc01a070 con 0x7f910c151f50 2024-04-22T06:31:48.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.009+0000 7f91097fa640 1 --2- 172.21.15.44:0/1119936576 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f90ec06c8f0 0x7f90ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:48.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.009+0000 7f91097fa640 1 -- 172.21.15.44:0/1119936576 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f90fc08b1e0 con 0x7f910c151f50 2024-04-22T06:31:48.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.009+0000 7f91109f8640 1 --2- 172.21.15.44:0/1119936576 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f90ec06c8f0 0x7f90ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:48.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.009+0000 7f91109f8640 1 --2- 172.21.15.44:0/1119936576 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f90ec06c8f0 0x7f90ec06edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9104011340 tx=0x7f910400b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:48.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.014+0000 7f91097fa640 1 -- 172.21.15.44:0/1119936576 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f910c158db0 con 0x7f910c151f50 2024-04-22T06:31:48.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.248+0000 7f9112c5c640 1 -- 172.21.15.44:0/1119936576 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f910c0c4a90 con 0x7f910c151f50 2024-04-22T06:31:48.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.249+0000 7f91097fa640 1 -- 172.21.15.44:0/1119936576 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7f90fc058c50 con 0x7f910c151f50 2024-04-22T06:31:48.248 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:31:48.248 INFO:teuthology.orchestra.run.smithi044.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-04-22T06:31:48.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.251+0000 7f9112c5c640 1 -- 172.21.15.44:0/1119936576 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f90ec06c8f0 msgr2=0x7f90ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:48.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.251+0000 7f9112c5c640 1 --2- 172.21.15.44:0/1119936576 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f90ec06c8f0 0x7f90ec06edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9104011340 tx=0x7f910400b040 comp rx=0 tx=0).stop 2024-04-22T06:31:48.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.251+0000 7f9112c5c640 1 -- 172.21.15.44:0/1119936576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f910c151f50 msgr2=0x7f910c0733b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:48.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.251+0000 7f9112c5c640 1 --2- 172.21.15.44:0/1119936576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f910c151f50 0x7f910c0733b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f90fc00c400 tx=0x7f90fc00c8c0 comp rx=0 tx=0).stop 2024-04-22T06:31:48.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.251+0000 7f9112c5c640 1 -- 172.21.15.44:0/1119936576 shutdown_connections 2024-04-22T06:31:48.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.251+0000 7f9112c5c640 1 --2- 172.21.15.44:0/1119936576 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f90ec06c8f0 0x7f90ec06edb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:48.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.251+0000 7f9112c5c640 1 --2- 172.21.15.44:0/1119936576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f910c151f50 0x7f910c0733b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:48.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.251+0000 7f9112c5c640 1 --2- 172.21.15.44:0/1119936576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f910c0cc980 0x7f910c0724d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:48.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.251+0000 7f9112c5c640 1 --2- 172.21.15.44:0/1119936576 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f910c0c8410 0x7f910c0c6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:48.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.251+0000 7f9112c5c640 1 -- 172.21.15.44:0/1119936576 >> 172.21.15.44:0/1119936576 conn(0x7f910c06d530 msgr2=0x7f910c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:48.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.251+0000 7f9112c5c640 1 -- 172.21.15.44:0/1119936576 shutdown_connections 2024-04-22T06:31:48.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:48.251+0000 7f9112c5c640 1 -- 172.21.15.44:0/1119936576 wait complete. 2024-04-22T06:31:51.261 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-22T06:31:51.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.375+0000 7fb1266d2640 1 Processor -- start 2024-04-22T06:31:51.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.375+0000 7fb1266d2640 1 -- start start 2024-04-22T06:31:51.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.376+0000 7fb1266d2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb120155570 0x7fb1200cb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:51.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.376+0000 7fb1266d2640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb1200cc8e0 0x7fb120165ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:51.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.376+0000 7fb1266d2640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1200ccd30 con 0x7fb1200cc4a0 2024-04-22T06:31:51.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.376+0000 7fb1266d2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fb12006c8a0 con 0x7fb120155570 2024-04-22T06:31:51.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.376+0000 7fb1266d2640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb12015c1d0 con 0x7fb1200cc8e0 2024-04-22T06:31:51.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.376+0000 7fb11f7fe640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fb1200cc4a0 0x7fb12006cd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:32882/0 (socket says 172.21.15.44:32882) 2024-04-22T06:31:51.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.376+0000 7fb11f7fe640 1 -- 172.21.15.44:0/2047028380 learned_addr learned my addr 172.21.15.44:0/2047028380 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:51.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.376+0000 7fb11ffff640 1 --2- 172.21.15.44:0/2047028380 >> v2:172.21.15.70:3300/0 conn(0x7fb120155570 0x7fb1200cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:51.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.376+0000 7fb124c6f640 1 --2- 172.21.15.44:0/2047028380 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb1200cc8e0 0x7fb120165ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:51.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.376+0000 7fb11effd640 1 -- 172.21.15.44:0/2047028380 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2205169166 0 0) 0x7fb1200ccd30 con 0x7fb1200cc4a0 2024-04-22T06:31:51.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.376+0000 7fb11effd640 1 -- 172.21.15.44:0/2047028380 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb10c003520 con 0x7fb1200cc4a0 2024-04-22T06:31:51.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.376+0000 7fb11ffff640 1 -- 172.21.15.44:0/2047028380 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb1200cc8e0 msgr2=0x7fb120165ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:51.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.376+0000 7fb11ffff640 1 --2- 172.21.15.44:0/2047028380 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb1200cc8e0 0x7fb120165ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:51.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.376+0000 7fb11ffff640 1 -- 172.21.15.44:0/2047028380 >> v1:172.21.15.44:6789/0 conn(0x7fb1200cc4a0 legacy=0x7fb12006cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:51.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.376+0000 7fb11ffff640 1 -- 172.21.15.44:0/2047028380 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb12015b650 con 0x7fb120155570 2024-04-22T06:31:51.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.376+0000 7fb124c6f640 1 --2- 172.21.15.44:0/2047028380 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb1200cc8e0 0x7fb120165ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:31:51.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.377+0000 7fb11ffff640 1 --2- 172.21.15.44:0/2047028380 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb120155570 0x7fb1200cb990 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb1100097a0 tx=0x7fb11002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cf677283e38ed0a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:51.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.377+0000 7fb11effd640 1 -- 172.21.15.44:0/2047028380 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1100315a0 con 0x7fb120155570 2024-04-22T06:31:51.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.377+0000 7fb11effd640 1 -- 172.21.15.44:0/2047028380 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb110037070 con 0x7fb120155570 2024-04-22T06:31:51.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.377+0000 7fb11effd640 1 -- 172.21.15.44:0/2047028380 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb110031940 con 0x7fb120155570 2024-04-22T06:31:51.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.378+0000 7fb1266d2640 1 -- 172.21.15.44:0/2047028380 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb120155570 msgr2=0x7fb1200cb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:51.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.378+0000 7fb1266d2640 1 --2- 172.21.15.44:0/2047028380 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb120155570 0x7fb1200cb990 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb1100097a0 tx=0x7fb11002b840 comp rx=0 tx=0).stop 2024-04-22T06:31:51.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.378+0000 7fb1266d2640 1 -- 172.21.15.44:0/2047028380 shutdown_connections 2024-04-22T06:31:51.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.378+0000 7fb1266d2640 1 --2- 172.21.15.44:0/2047028380 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb1200cc8e0 0x7fb120165ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:51.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.378+0000 7fb1266d2640 1 --2- 172.21.15.44:0/2047028380 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb120155570 0x7fb1200cb990 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:51.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.378+0000 7fb1266d2640 1 -- 172.21.15.44:0/2047028380 >> 172.21.15.44:0/2047028380 conn(0x7fb120075870 msgr2=0x7fb120073020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:51.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.378+0000 7fb1266d2640 1 -- 172.21.15.44:0/2047028380 shutdown_connections 2024-04-22T06:31:51.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.378+0000 7fb1266d2640 1 -- 172.21.15.44:0/2047028380 wait complete. 2024-04-22T06:31:51.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.379+0000 7fb1266d2640 1 Processor -- start 2024-04-22T06:31:51.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.379+0000 7fb1266d2640 1 -- start start 2024-04-22T06:31:51.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.380+0000 7fb1266d2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1200cc4a0 0x7fb120155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:51.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.380+0000 7fb1266d2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1200cc8e0 0x7fb1201f4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:51.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.380+0000 7fb1266d2640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb120155570 0x7fb1200c8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:51.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.380+0000 7fb1266d2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb12015bef0 con 0x7fb1200cc8e0 2024-04-22T06:31:51.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.380+0000 7fb1266d2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb12015bc10 con 0x7fb1200cc4a0 2024-04-22T06:31:51.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.380+0000 7fb1266d2640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb120155950 con 0x7fb120155570 2024-04-22T06:31:51.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.380+0000 7fb11f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1200cc8e0 0x7fb1201f4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:51.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.380+0000 7fb11ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1200cc4a0 0x7fb120155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:51.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.380+0000 7fb11f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1200cc8e0 0x7fb1201f4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48648/0 (socket says 172.21.15.44:48648) 2024-04-22T06:31:51.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.380+0000 7fb11ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1200cc4a0 0x7fb120155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:49426/0 (socket says 172.21.15.44:49426) 2024-04-22T06:31:51.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.380+0000 7fb124c6f640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb120155570 0x7fb1200c8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:51.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.380+0000 7fb11f7fe640 1 -- 172.21.15.44:0/4294474430 learned_addr learned my addr 172.21.15.44:0/4294474430 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:51.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.381+0000 7fb11f7fe640 1 -- 172.21.15.44:0/4294474430 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb120155570 msgr2=0x7fb1200c8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:51.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.381+0000 7fb11f7fe640 1 --2- 172.21.15.44:0/4294474430 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb120155570 0x7fb1200c8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:51.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.381+0000 7fb11f7fe640 1 -- 172.21.15.44:0/4294474430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1200cc4a0 msgr2=0x7fb120155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:51.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.381+0000 7fb11f7fe640 1 --2- 172.21.15.44:0/4294474430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1200cc4a0 0x7fb120155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:51.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.381+0000 7fb11f7fe640 1 -- 172.21.15.44:0/4294474430 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb12015c390 con 0x7fb1200cc8e0 2024-04-22T06:31:51.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.381+0000 7fb11ffff640 1 --2- 172.21.15.44:0/4294474430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1200cc4a0 0x7fb120155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:31:51.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.381+0000 7fb11f7fe640 1 --2- 172.21.15.44:0/4294474430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1200cc8e0 0x7fb1201f4c40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb1080010e0 tx=0x7fb10800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:51.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.381+0000 7fb11cff9640 1 -- 172.21.15.44:0/4294474430 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb108017070 con 0x7fb1200cc8e0 2024-04-22T06:31:51.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.381+0000 7fb11cff9640 1 -- 172.21.15.44:0/4294474430 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb108005d00 con 0x7fb1200cc8e0 2024-04-22T06:31:51.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.381+0000 7fb1266d2640 1 -- 172.21.15.44:0/4294474430 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb12015c630 con 0x7fb1200cc8e0 2024-04-22T06:31:51.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.381+0000 7fb11cff9640 1 -- 172.21.15.44:0/4294474430 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb108007310 con 0x7fb1200cc8e0 2024-04-22T06:31:51.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.381+0000 7fb1266d2640 1 -- 172.21.15.44:0/4294474430 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb12015c8f0 con 0x7fb1200cc8e0 2024-04-22T06:31:51.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.382+0000 7fb1266d2640 1 -- 172.21.15.44:0/4294474430 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb120068cc0 con 0x7fb1200cc8e0 2024-04-22T06:31:51.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.383+0000 7fb11cff9640 1 -- 172.21.15.44:0/4294474430 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb10801a070 con 0x7fb1200cc8e0 2024-04-22T06:31:51.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.383+0000 7fb11cff9640 1 --2- 172.21.15.44:0/4294474430 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb10006c920 0x7fb10006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:51.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.383+0000 7fb11cff9640 1 -- 172.21.15.44:0/4294474430 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fb10808b2e0 con 0x7fb1200cc8e0 2024-04-22T06:31:51.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.383+0000 7fb11ffff640 1 --2- 172.21.15.44:0/4294474430 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb10006c920 0x7fb10006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:51.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.384+0000 7fb11ffff640 1 --2- 172.21.15.44:0/4294474430 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb10006c920 0x7fb10006ede0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb110036000 tx=0x7fb11003d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:51.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.388+0000 7fb11cff9640 1 -- 172.21.15.44:0/4294474430 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb108058c30 con 0x7fb1200cc8e0 2024-04-22T06:31:51.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.610+0000 7fb1266d2640 1 -- 172.21.15.44:0/4294474430 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb1200b63d0 con 0x7fb1200cc8e0 2024-04-22T06:31:51.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.611+0000 7fb11cff9640 1 -- 172.21.15.44:0/4294474430 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fb10805cf30 con 0x7fb1200cc8e0 2024-04-22T06:31:51.610 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:31:51.610 INFO:teuthology.orchestra.run.smithi044.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-22T06:31:51.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.613+0000 7fb1266d2640 1 -- 172.21.15.44:0/4294474430 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb10006c920 msgr2=0x7fb10006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:51.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.613+0000 7fb1266d2640 1 --2- 172.21.15.44:0/4294474430 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb10006c920 0x7fb10006ede0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb110036000 tx=0x7fb11003d040 comp rx=0 tx=0).stop 2024-04-22T06:31:51.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.613+0000 7fb1266d2640 1 -- 172.21.15.44:0/4294474430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1200cc8e0 msgr2=0x7fb1201f4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:51.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.613+0000 7fb1266d2640 1 --2- 172.21.15.44:0/4294474430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1200cc8e0 0x7fb1201f4c40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb1080010e0 tx=0x7fb10800c8a0 comp rx=0 tx=0).stop 2024-04-22T06:31:51.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.613+0000 7fb1266d2640 1 -- 172.21.15.44:0/4294474430 shutdown_connections 2024-04-22T06:31:51.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.613+0000 7fb1266d2640 1 --2- 172.21.15.44:0/4294474430 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb10006c920 0x7fb10006ede0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:51.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.613+0000 7fb1266d2640 1 --2- 172.21.15.44:0/4294474430 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb120155570 0x7fb1200c8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:51.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.613+0000 7fb1266d2640 1 --2- 172.21.15.44:0/4294474430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1200cc8e0 0x7fb1201f4c40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:51.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.613+0000 7fb1266d2640 1 --2- 172.21.15.44:0/4294474430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1200cc4a0 0x7fb120155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:51.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.613+0000 7fb1266d2640 1 -- 172.21.15.44:0/4294474430 >> 172.21.15.44:0/4294474430 conn(0x7fb120075870 msgr2=0x7fb120072a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:51.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.613+0000 7fb1266d2640 1 -- 172.21.15.44:0/4294474430 shutdown_connections 2024-04-22T06:31:51.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.613+0000 7fb1266d2640 1 -- 172.21.15.44:0/4294474430 wait complete. 2024-04-22T06:31:51.622 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-22T06:31:51.622 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T06:31:51.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.724+0000 7f98df811640 1 Processor -- start 2024-04-22T06:31:51.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.724+0000 7f98df811640 1 -- start start 2024-04-22T06:31:51.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.725+0000 7f98df811640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f98d8074f20 0x7f98d8072c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:51.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.725+0000 7f98df811640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f98d8075300 0x7f98d8070490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:51.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.725+0000 7f98df811640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98d80709d0 0x7f98d8143f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:51.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.725+0000 7f98df811640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f98d815f5f0 con 0x7f98d8075300 2024-04-22T06:31:51.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.725+0000 7f98df811640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f98d80c8370 con 0x7f98d8074f20 2024-04-22T06:31:51.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.725+0000 7f98df811640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f98d815e720 con 0x7f98d80709d0 2024-04-22T06:31:51.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.725+0000 7f98dcdac640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f98d8075300 0x7f98d8070490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:51.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.725+0000 7f98dcdac640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f98d8075300 0x7f98d8070490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48654/0 (socket says 172.21.15.44:48654) 2024-04-22T06:31:51.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.725+0000 7f98dcdac640 1 -- 172.21.15.44:0/1906332820 learned_addr learned my addr 172.21.15.44:0/1906332820 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:51.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.725+0000 7f98dd5ad640 1 --2- 172.21.15.44:0/1906332820 >> v2:172.21.15.70:3300/0 conn(0x7f98d8074f20 0x7f98d8072c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:51.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.725+0000 7f98dddae640 1 --2- 172.21.15.44:0/1906332820 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98d80709d0 0x7f98d8143f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:51.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.725+0000 7f98dcdac640 1 -- 172.21.15.44:0/1906332820 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98d80709d0 msgr2=0x7f98d8143f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:51.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.725+0000 7f98dcdac640 1 --2- 172.21.15.44:0/1906332820 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98d80709d0 0x7f98d8143f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:51.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.725+0000 7f98dcdac640 1 -- 172.21.15.44:0/1906332820 >> v2:172.21.15.70:3300/0 conn(0x7f98d8074f20 msgr2=0x7f98d8072c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:51.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.725+0000 7f98dcdac640 1 --2- 172.21.15.44:0/1906332820 >> v2:172.21.15.70:3300/0 conn(0x7f98d8074f20 0x7f98d8072c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:51.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.725+0000 7f98dcdac640 1 -- 172.21.15.44:0/1906332820 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d815e9e0 con 0x7f98d8075300 2024-04-22T06:31:51.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.725+0000 7f98dd5ad640 1 --2- 172.21.15.44:0/1906332820 >> v2:172.21.15.70:3300/0 conn(0x7f98d8074f20 0x7f98d8072c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:31:51.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.726+0000 7f98dcdac640 1 --2- 172.21.15.44:0/1906332820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98d8075300 0x7f98d8070490 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f98d806fbb0 tx=0x7f98c802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2043cd8c05fb795 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:51.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.726+0000 7f98cffff640 1 -- 172.21.15.44:0/1906332820 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c8034070 con 0x7f98d8075300 2024-04-22T06:31:51.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.726+0000 7f98cffff640 1 -- 172.21.15.44:0/1906332820 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98c8030860 con 0x7f98d8075300 2024-04-22T06:31:51.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.726+0000 7f98cffff640 1 -- 172.21.15.44:0/1906332820 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c8030c30 con 0x7f98d8075300 2024-04-22T06:31:51.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.727+0000 7f98df811640 1 -- 172.21.15.44:0/1906332820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98d8075300 msgr2=0x7f98d8070490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:51.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.727+0000 7f98df811640 1 --2- 172.21.15.44:0/1906332820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98d8075300 0x7f98d8070490 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f98d806fbb0 tx=0x7f98c802b9a0 comp rx=0 tx=0).stop 2024-04-22T06:31:51.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.727+0000 7f98df811640 1 -- 172.21.15.44:0/1906332820 shutdown_connections 2024-04-22T06:31:51.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.727+0000 7f98df811640 1 --2- 172.21.15.44:0/1906332820 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98d80709d0 0x7f98d8143f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:51.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.727+0000 7f98df811640 1 --2- 172.21.15.44:0/1906332820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98d8075300 0x7f98d8070490 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:51.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.727+0000 7f98df811640 1 --2- 172.21.15.44:0/1906332820 >> v2:172.21.15.70:3300/0 conn(0x7f98d8074f20 0x7f98d8072c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:51.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.727+0000 7f98df811640 1 -- 172.21.15.44:0/1906332820 >> 172.21.15.44:0/1906332820 conn(0x7f98d80ca330 msgr2=0x7f98d80ca740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:51.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.727+0000 7f98df811640 1 -- 172.21.15.44:0/1906332820 shutdown_connections 2024-04-22T06:31:51.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.727+0000 7f98df811640 1 -- 172.21.15.44:0/1906332820 wait complete. 2024-04-22T06:31:51.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.728+0000 7f98df811640 1 Processor -- start 2024-04-22T06:31:51.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.728+0000 7f98df811640 1 -- start start 2024-04-22T06:31:51.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.729+0000 7f98df811640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98d80709d0 0x7f98d80cc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:51.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.729+0000 7f98df811640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98d8074f20 0x7f98d8155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:51.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.729+0000 7f98df811640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98d8075300 0x7f98d8156f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:51.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.729+0000 7f98df811640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f98d815f280 con 0x7f98d80709d0 2024-04-22T06:31:51.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.729+0000 7f98df811640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f98d815efa0 con 0x7f98d8074f20 2024-04-22T06:31:51.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.729+0000 7f98df811640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f98d81444b0 con 0x7f98d8075300 2024-04-22T06:31:51.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.729+0000 7f98dd5ad640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98d80709d0 0x7f98d80cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:51.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.729+0000 7f98dd5ad640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98d80709d0 0x7f98d80cc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48664/0 (socket says 172.21.15.44:48664) 2024-04-22T06:31:51.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.729+0000 7f98dd5ad640 1 -- 172.21.15.44:0/513485764 learned_addr learned my addr 172.21.15.44:0/513485764 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:51.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.729+0000 7f98dcdac640 1 --2- 172.21.15.44:0/513485764 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98d8074f20 0x7f98d8155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:51.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.729+0000 7f98dddae640 1 --2- 172.21.15.44:0/513485764 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98d8075300 0x7f98d8156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:51.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.729+0000 7f98dd5ad640 1 -- 172.21.15.44:0/513485764 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98d8075300 msgr2=0x7f98d8156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:51.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.729+0000 7f98dd5ad640 1 --2- 172.21.15.44:0/513485764 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98d8075300 0x7f98d8156f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:51.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.729+0000 7f98dd5ad640 1 -- 172.21.15.44:0/513485764 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98d8074f20 msgr2=0x7f98d8155fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:51.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.729+0000 7f98dd5ad640 1 --2- 172.21.15.44:0/513485764 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98d8074f20 0x7f98d8155fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:51.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.729+0000 7f98dd5ad640 1 -- 172.21.15.44:0/513485764 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d81568d0 con 0x7f98d80709d0 2024-04-22T06:31:51.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.729+0000 7f98dd5ad640 1 --2- 172.21.15.44:0/513485764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98d80709d0 0x7f98d80cc960 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f98d400ebe0 tx=0x7f98d400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:51.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.729+0000 7f98cdffb640 1 -- 172.21.15.44:0/513485764 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d4017070 con 0x7f98d80709d0 2024-04-22T06:31:51.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.729+0000 7f98cdffb640 1 -- 172.21.15.44:0/513485764 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98d400cc30 con 0x7f98d80709d0 2024-04-22T06:31:51.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.729+0000 7f98df811640 1 -- 172.21.15.44:0/513485764 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98d81f4630 con 0x7f98d80709d0 2024-04-22T06:31:51.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.729+0000 7f98cdffb640 1 -- 172.21.15.44:0/513485764 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d4013470 con 0x7f98d80709d0 2024-04-22T06:31:51.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.730+0000 7f98df811640 1 -- 172.21.15.44:0/513485764 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98d81f4bf0 con 0x7f98d80709d0 2024-04-22T06:31:51.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.731+0000 7f98df811640 1 -- 172.21.15.44:0/513485764 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98d8068cc0 con 0x7f98d80709d0 2024-04-22T06:31:51.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.731+0000 7f98cdffb640 1 -- 172.21.15.44:0/513485764 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f98d401a070 con 0x7f98d80709d0 2024-04-22T06:31:51.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.731+0000 7f98cdffb640 1 --2- 172.21.15.44:0/513485764 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f98bc06c920 0x7f98bc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:51.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.732+0000 7f98cdffb640 1 -- 172.21.15.44:0/513485764 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f98d408c0c0 con 0x7f98d80709d0 2024-04-22T06:31:51.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.732+0000 7f98dcdac640 1 --2- 172.21.15.44:0/513485764 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f98bc06c920 0x7f98bc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:51.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.732+0000 7f98dcdac640 1 --2- 172.21.15.44:0/513485764 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f98bc06c920 0x7f98bc06ede0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f98c8036000 tx=0x7f98c803b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:51.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.736+0000 7f98cdffb640 1 -- 172.21.15.44:0/513485764 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f98d4059b30 con 0x7f98d80709d0 2024-04-22T06:31:51.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.920+0000 7f98df811640 1 -- 172.21.15.44:0/513485764 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f98d806a090 con 0x7f98d80709d0 2024-04-22T06:31:51.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.921+0000 7f98cdffb640 1 -- 172.21.15.44:0/513485764 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18794 (secure 0 0 0) 0x7f98d405de30 con 0x7f98d80709d0 2024-04-22T06:31:51.920 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:31:51.921 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":27,"fsid":"9b0a34ca-403d-4513-8a66-d3749ba559aa","created":"2024-04-22T06:30:42.334581+0000","modified":"2024-04-22T06:31:48.138913+0000","last_up_change":"2024-04-22T06:30:52.942257+0000","last_in_change":"2024-04-22T06:30:44.467727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T06:30:54.560256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T06:31:04.612387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T06:31:05.312124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-22T06:31:09.025550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b4a6692-53c9-472d-b2f3-1dd29a6679ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6817","nonce":2220268493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6819","nonce":2220268493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6823","nonce":2220268493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6821","nonce":2220268493}]},"public_addr":"172.21.15.44:6817/2220268493","cluster_addr":"172.21.15.44:6819/2220268493","heartbeat_back_addr":"172.21.15.44:6823/2220268493","heartbeat_front_addr":"172.21.15.44:6821/2220268493","state":["exists","up"]},{"osd":1,"uuid":"70a6370a-7c0a-4d64-a0e4-b2e726b4f291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6809","nonce":1278678977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6811","nonce":1278678977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6815","nonce":1278678977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6813","nonce":1278678977}]},"public_addr":"172.21.15.44:6809/1278678977","cluster_addr":"172.21.15.44:6811/1278678977","heartbeat_back_addr":"172.21.15.44:6815/1278678977","heartbeat_front_addr":"172.21.15.44:6813/1278678977","state":["exists","up"]},{"osd":2,"uuid":"f2e51a62-ca91-470d-bd5a-5710c61deee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6801","nonce":2544087903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6803","nonce":2544087903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6807","nonce":2544087903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6805","nonce":2544087903}]},"public_addr":"172.21.15.44:6801/2544087903","cluster_addr":"172.21.15.44:6803/2544087903","heartbeat_back_addr":"172.21.15.44:6807/2544087903","heartbeat_front_addr":"172.21.15.44:6805/2544087903","state":["exists","up"]},{"osd":3,"uuid":"bc514203-9af9-4f1f-b18e-765bdd5de6e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6825","nonce":1692928109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6827","nonce":1692928109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6831","nonce":1692928109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6829","nonce":1692928109}]},"public_addr":"172.21.15.44:6825/1692928109","cluster_addr":"172.21.15.44:6827/1692928109","heartbeat_back_addr":"172.21.15.44:6831/1692928109","heartbeat_front_addr":"172.21.15.44:6829/1692928109","state":["exists","up"]},{"osd":4,"uuid":"357c439a-6e08-4c19-ac18-b68e6c890497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6820","nonce":3802577427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6824","nonce":3802577427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6831","nonce":3802577427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6828","nonce":3802577427}]},"public_addr":"172.21.15.70:6820/3802577427","cluster_addr":"172.21.15.70:6824/3802577427","heartbeat_back_addr":"172.21.15.70:6831/3802577427","heartbeat_front_addr":"172.21.15.70:6828/3802577427","state":["exists","up"]},{"osd":5,"uuid":"952fb88f-d56e-4305-bfa8-d32674332dfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6801","nonce":664757345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6803","nonce":664757345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6807","nonce":664757345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6805","nonce":664757345}]},"public_addr":"172.21.15.70:6801/664757345","cluster_addr":"172.21.15.70:6803/664757345","heartbeat_back_addr":"172.21.15.70:6807/664757345","heartbeat_front_addr":"172.21.15.70:6805/664757345","state":["exists","up"]},{"osd":6,"uuid":"74399b3d-fac2-45ab-9172-2cd95b02450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6817","nonce":1623253482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6821","nonce":1623253482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6827","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6829","nonce":1623253482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6825","nonce":1623253482}]},"public_addr":"172.21.15.70:6817/1623253482","cluster_addr":"172.21.15.70:6821/1623253482","heartbeat_back_addr":"172.21.15.70:6829/1623253482","heartbeat_front_addr":"172.21.15.70:6825/1623253482","state":["exists","up"]},{"osd":7,"uuid":"8f35508d-6fcc-43ff-8ee7-0498a7ab3b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6809","nonce":1551180300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6811","nonce":1551180300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6815","nonce":1551180300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6813","nonce":1551180300}]},"public_addr":"172.21.15.70:6809/1551180300","cluster_addr":"172.21.15.70:6811/1551180300","heartbeat_back_addr":"172.21.15.70:6815/1551180300","heartbeat_front_addr":"172.21.15.70:6813/1551180300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.612502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.653756+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.521169+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.649955+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.640231+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.686638+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.642199+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.700360+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T06:31:51.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.923+0000 7f98df811640 1 -- 172.21.15.44:0/513485764 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f98bc06c920 msgr2=0x7f98bc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:51.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.923+0000 7f98df811640 1 --2- 172.21.15.44:0/513485764 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f98bc06c920 0x7f98bc06ede0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f98c8036000 tx=0x7f98c803b040 comp rx=0 tx=0).stop 2024-04-22T06:31:51.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.923+0000 7f98df811640 1 -- 172.21.15.44:0/513485764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98d80709d0 msgr2=0x7f98d80cc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:51.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.923+0000 7f98df811640 1 --2- 172.21.15.44:0/513485764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98d80709d0 0x7f98d80cc960 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f98d400ebe0 tx=0x7f98d400c520 comp rx=0 tx=0).stop 2024-04-22T06:31:51.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.923+0000 7f98df811640 1 -- 172.21.15.44:0/513485764 shutdown_connections 2024-04-22T06:31:51.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.923+0000 7f98df811640 1 --2- 172.21.15.44:0/513485764 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f98bc06c920 0x7f98bc06ede0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:51.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.923+0000 7f98df811640 1 --2- 172.21.15.44:0/513485764 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98d8075300 0x7f98d8156f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:51.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.923+0000 7f98df811640 1 --2- 172.21.15.44:0/513485764 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98d8074f20 0x7f98d8155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:51.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.923+0000 7f98df811640 1 --2- 172.21.15.44:0/513485764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98d80709d0 0x7f98d80cc960 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:51.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.923+0000 7f98df811640 1 -- 172.21.15.44:0/513485764 >> 172.21.15.44:0/513485764 conn(0x7f98d80ca330 msgr2=0x7f98d80ca710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:51.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.924+0000 7f98df811640 1 -- 172.21.15.44:0/513485764 shutdown_connections 2024-04-22T06:31:51.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:51.924+0000 7f98df811640 1 -- 172.21.15.44:0/513485764 wait complete. 2024-04-22T06:31:51.933 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T06:30:54.560256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T06:31:04.612387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T06:31:05.312124+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-22T06:31:09.025550+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-22T06:31:51.934 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T06:31:52.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.038+0000 7f6e90481640 1 Processor -- start 2024-04-22T06:31:52.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.038+0000 7f6e90481640 1 -- start start 2024-04-22T06:31:52.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.039+0000 7f6e90481640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f6e880c6150 0x7f6e88158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:52.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.039+0000 7f6e90481640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e88073c90 con 0x7f6e880c5d70 2024-04-22T06:31:52.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.039+0000 7f6e90481640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e88073f70 con 0x7f6e880c8750 2024-04-22T06:31:52.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.039+0000 7f6e90481640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f6e880cf630 con 0x7f6e880c6150 2024-04-22T06:31:52.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.039+0000 7f6e8da1c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f6e880c6150 0x7f6e88158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:52.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.039+0000 7f6e8e21d640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f6e880c5d70 0x7f6e880731e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:38644/0 (socket says 172.21.15.44:38644) 2024-04-22T06:31:52.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.039+0000 7f6e8e21d640 1 -- 172.21.15.44:0/1704423920 learned_addr learned my addr 172.21.15.44:0/1704423920 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:52.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.039+0000 7f6e8da1c640 1 -- 172.21.15.44:0/1704423920 >> v1:172.21.15.70:6789/0 conn(0x7f6e880c8750 legacy=0x7f6e88152dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:52.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.039+0000 7f6e8da1c640 1 -- 172.21.15.44:0/1704423920 >> v1:172.21.15.44:6789/0 conn(0x7f6e880c5d70 legacy=0x7f6e880731e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:52.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.039+0000 7f6e8d21b640 1 -- 172.21.15.44:0/1704423920 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3685429110 0 0) 0x7f6e88073f70 con 0x7f6e880c8750 2024-04-22T06:31:52.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.039+0000 7f6e8da1c640 1 -- 172.21.15.44:0/1704423920 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e880c7c70 con 0x7f6e880c6150 2024-04-22T06:31:52.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.040+0000 7f6e8da1c640 1 --2- 172.21.15.44:0/1704423920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e880c6150 0x7f6e88158ff0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c0097d0 tx=0x7f6e7c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1290ebf586a430d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:52.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.040+0000 7f6e8d21b640 1 -- 172.21.15.44:0/1704423920 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c038070 con 0x7f6e880c6150 2024-04-22T06:31:52.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.040+0000 7f6e8d21b640 1 -- 172.21.15.44:0/1704423920 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e7c031700 con 0x7f6e880c6150 2024-04-22T06:31:52.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.040+0000 7f6e8d21b640 1 -- 172.21.15.44:0/1704423920 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c031ad0 con 0x7f6e880c6150 2024-04-22T06:31:52.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.041+0000 7f6e90481640 1 -- 172.21.15.44:0/1704423920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e880c6150 msgr2=0x7f6e88158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:52.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.041+0000 7f6e90481640 1 --2- 172.21.15.44:0/1704423920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e880c6150 0x7f6e88158ff0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c0097d0 tx=0x7f6e7c02b820 comp rx=0 tx=0).stop 2024-04-22T06:31:52.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.041+0000 7f6e90481640 1 -- 172.21.15.44:0/1704423920 shutdown_connections 2024-04-22T06:31:52.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.041+0000 7f6e90481640 1 --2- 172.21.15.44:0/1704423920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e880c6150 0x7f6e88158ff0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:52.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.041+0000 7f6e90481640 1 -- 172.21.15.44:0/1704423920 >> 172.21.15.44:0/1704423920 conn(0x7f6e880ce700 msgr2=0x7f6e880ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:52.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.041+0000 7f6e90481640 1 -- 172.21.15.44:0/1704423920 shutdown_connections 2024-04-22T06:31:52.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.041+0000 7f6e90481640 1 -- 172.21.15.44:0/1704423920 wait complete. 2024-04-22T06:31:52.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.042+0000 7f6e90481640 1 Processor -- start 2024-04-22T06:31:52.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.042+0000 7f6e90481640 1 -- start start 2024-04-22T06:31:52.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.043+0000 7f6e90481640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e880c5d70 0x7f6e881ef680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:52.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.043+0000 7f6e90481640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6e880c6150 0x7f6e880cd260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:52.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.043+0000 7f6e90481640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e880c8750 0x7f6e881df0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:52.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.043+0000 7f6e90481640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6e880738f0 con 0x7f6e880c5d70 2024-04-22T06:31:52.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.043+0000 7f6e90481640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6e880c8230 con 0x7f6e880c8750 2024-04-22T06:31:52.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.043+0000 7f6e90481640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f6e88073a70 con 0x7f6e880c6150 2024-04-22T06:31:52.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.043+0000 7f6e8e21d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e880c5d70 0x7f6e881ef680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:52.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.043+0000 7f6e8e21d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e880c5d70 0x7f6e881ef680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54930/0 (socket says 172.21.15.44:54930) 2024-04-22T06:31:52.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.043+0000 7f6e8e21d640 1 -- 172.21.15.44:0/967430489 learned_addr learned my addr 172.21.15.44:0/967430489 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:52.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.043+0000 7f6e8da1c640 1 --2- 172.21.15.44:0/967430489 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6e880c6150 0x7f6e880cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:52.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.043+0000 7f6e8ea1e640 1 --2- 172.21.15.44:0/967430489 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e880c8750 0x7f6e881df0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:52.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.043+0000 7f6e8e21d640 1 -- 172.21.15.44:0/967430489 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6e880c6150 msgr2=0x7f6e880cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:52.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.043+0000 7f6e8e21d640 1 --2- 172.21.15.44:0/967430489 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6e880c6150 0x7f6e880cd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:52.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.043+0000 7f6e8e21d640 1 -- 172.21.15.44:0/967430489 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e880c8750 msgr2=0x7f6e881df0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:52.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.043+0000 7f6e8e21d640 1 --2- 172.21.15.44:0/967430489 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e880c8750 0x7f6e881df0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:52.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.043+0000 7f6e8e21d640 1 -- 172.21.15.44:0/967430489 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e881dfc60 con 0x7f6e880c5d70 2024-04-22T06:31:52.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.043+0000 7f6e8da1c640 1 --2- 172.21.15.44:0/967430489 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6e880c6150 0x7f6e880cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:31:52.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.043+0000 7f6e8ea1e640 1 --2- 172.21.15.44:0/967430489 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e880c8750 0x7f6e881df0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:31:52.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.043+0000 7f6e8e21d640 1 --2- 172.21.15.44:0/967430489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e880c5d70 0x7f6e881ef680 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6e8000c7b0 tx=0x7f6e8000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:52.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.044+0000 7f6e86ffd640 1 -- 172.21.15.44:0/967430489 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e80018070 con 0x7f6e880c5d70 2024-04-22T06:31:52.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.044+0000 7f6e86ffd640 1 -- 172.21.15.44:0/967430489 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e800072d0 con 0x7f6e880c5d70 2024-04-22T06:31:52.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.044+0000 7f6e90481640 1 -- 172.21.15.44:0/967430489 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e881dff40 con 0x7f6e880c5d70 2024-04-22T06:31:52.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.044+0000 7f6e86ffd640 1 -- 172.21.15.44:0/967430489 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e800047e0 con 0x7f6e880c5d70 2024-04-22T06:31:52.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.044+0000 7f6e90481640 1 -- 172.21.15.44:0/967430489 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e881e0500 con 0x7f6e880c5d70 2024-04-22T06:31:52.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.044+0000 7f6e90481640 1 -- 172.21.15.44:0/967430489 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e88069310 con 0x7f6e880c5d70 2024-04-22T06:31:52.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.045+0000 7f6e86ffd640 1 -- 172.21.15.44:0/967430489 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6e8001b070 con 0x7f6e880c5d70 2024-04-22T06:31:52.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.045+0000 7f6e86ffd640 1 --2- 172.21.15.44:0/967430489 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f6e7406c920 0x7f6e7406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:52.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.046+0000 7f6e86ffd640 1 -- 172.21.15.44:0/967430489 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f6e8008b350 con 0x7f6e880c5d70 2024-04-22T06:31:52.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.046+0000 7f6e8da1c640 1 --2- 172.21.15.44:0/967430489 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f6e7406c920 0x7f6e7406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:52.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.046+0000 7f6e8da1c640 1 --2- 172.21.15.44:0/967430489 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f6e7406c920 0x7f6e7406ede0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c035000 tx=0x7f6e7c008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:52.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.050+0000 7f6e86ffd640 1 -- 172.21.15.44:0/967430489 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6e80058d30 con 0x7f6e880c5d70 2024-04-22T06:31:52.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.236+0000 7f6e90481640 1 -- 172.21.15.44:0/967430489 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6e880743b0 con 0x7f6e880c5d70 2024-04-22T06:31:52.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.236+0000 7f6e86ffd640 1 -- 172.21.15.44:0/967430489 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f6e8005d030 con 0x7f6e880c5d70 2024-04-22T06:31:52.238 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-04-22T06:31:52.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.238+0000 7f6e90481640 1 -- 172.21.15.44:0/967430489 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f6e7406c920 msgr2=0x7f6e7406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:52.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.238+0000 7f6e90481640 1 --2- 172.21.15.44:0/967430489 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f6e7406c920 0x7f6e7406ede0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c035000 tx=0x7f6e7c008030 comp rx=0 tx=0).stop 2024-04-22T06:31:52.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.238+0000 7f6e90481640 1 -- 172.21.15.44:0/967430489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e880c5d70 msgr2=0x7f6e881ef680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:52.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.238+0000 7f6e90481640 1 --2- 172.21.15.44:0/967430489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e880c5d70 0x7f6e881ef680 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6e8000c7b0 tx=0x7f6e8000cc70 comp rx=0 tx=0).stop 2024-04-22T06:31:52.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.238+0000 7f6e90481640 1 -- 172.21.15.44:0/967430489 shutdown_connections 2024-04-22T06:31:52.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.238+0000 7f6e90481640 1 --2- 172.21.15.44:0/967430489 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f6e7406c920 0x7f6e7406ede0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:52.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.238+0000 7f6e90481640 1 --2- 172.21.15.44:0/967430489 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6e880c6150 0x7f6e880cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:52.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.238+0000 7f6e90481640 1 --2- 172.21.15.44:0/967430489 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e880c8750 0x7f6e881df0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:52.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.238+0000 7f6e90481640 1 --2- 172.21.15.44:0/967430489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e880c5d70 0x7f6e881ef680 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:52.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.238+0000 7f6e90481640 1 -- 172.21.15.44:0/967430489 >> 172.21.15.44:0/967430489 conn(0x7f6e880ce700 msgr2=0x7f6e88157f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:52.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.238+0000 7f6e90481640 1 -- 172.21.15.44:0/967430489 shutdown_connections 2024-04-22T06:31:52.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.238+0000 7f6e90481640 1 -- 172.21.15.44:0/967430489 wait complete. 2024-04-22T06:31:52.247 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T06:31:52.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.352+0000 7f0d2bc43640 1 Processor -- start 2024-04-22T06:31:52.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.353+0000 7f0d2bc43640 1 -- start start 2024-04-22T06:31:52.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.353+0000 7f0d2bc43640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f0d240c5d70 0x7f0d24158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:52.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.353+0000 7f0d2bc43640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0d240c6150 0x7f0d240c8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:52.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.353+0000 7f0d2bc43640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d24073c90 con 0x7f0d24070810 2024-04-22T06:31:52.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.353+0000 7f0d2bc43640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f0d240cf630 con 0x7f0d240c5d70 2024-04-22T06:31:52.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.353+0000 7f0d2bc43640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f0d24070c80 con 0x7f0d240c6150 2024-04-22T06:31:52.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.353+0000 7f0d299df640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f0d240c5d70 0x7f0d24158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:52.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.353+0000 7f0d299df640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f0d240c5d70 0x7f0d24158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54944/0 (socket says 172.21.15.44:54944) 2024-04-22T06:31:52.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.353+0000 7f0d299df640 1 -- 172.21.15.44:0/1972832146 learned_addr learned my addr 172.21.15.44:0/1972832146 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:52.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.353+0000 7f0d291de640 1 --2- 172.21.15.44:0/1972832146 >> v2:172.21.15.70:3300/0 conn(0x7f0d240c6150 0x7f0d240c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:52.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.354+0000 7f0d299df640 1 -- 172.21.15.44:0/1972832146 >> v2:172.21.15.70:3300/0 conn(0x7f0d240c6150 msgr2=0x7f0d240c8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:52.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.354+0000 7f0d299df640 1 --2- 172.21.15.44:0/1972832146 >> v2:172.21.15.70:3300/0 conn(0x7f0d240c6150 0x7f0d240c8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:52.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.354+0000 7f0d299df640 1 -- 172.21.15.44:0/1972832146 >> v1:172.21.15.70:6789/0 conn(0x7f0d24070810 legacy=0x7f0d240731e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:52.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.354+0000 7f0d299df640 1 -- 172.21.15.44:0/1972832146 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d24073f70 con 0x7f0d240c5d70 2024-04-22T06:31:52.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.355+0000 7f0d299df640 1 --2- 172.21.15.44:0/1972832146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d240c5d70 0x7f0d24158ff0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0d24050d80 tx=0x7f0d2002ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25fc6965a4e6121a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:52.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.355+0000 7f0d289dd640 1 -- 172.21.15.44:0/1972832146 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2002f850 con 0x7f0d240c5d70 2024-04-22T06:31:52.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.355+0000 7f0d289dd640 1 -- 172.21.15.44:0/1972832146 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d2002f9f0 con 0x7f0d240c5d70 2024-04-22T06:31:52.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.355+0000 7f0d289dd640 1 -- 172.21.15.44:0/1972832146 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2002fdc0 con 0x7f0d240c5d70 2024-04-22T06:31:52.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.355+0000 7f0d2bc43640 1 -- 172.21.15.44:0/1972832146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d240c5d70 msgr2=0x7f0d24158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:52.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.355+0000 7f0d2bc43640 1 --2- 172.21.15.44:0/1972832146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d240c5d70 0x7f0d24158ff0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0d24050d80 tx=0x7f0d2002ba30 comp rx=0 tx=0).stop 2024-04-22T06:31:52.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.355+0000 7f0d2bc43640 1 -- 172.21.15.44:0/1972832146 shutdown_connections 2024-04-22T06:31:52.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.355+0000 7f0d2bc43640 1 --2- 172.21.15.44:0/1972832146 >> v2:172.21.15.70:3300/0 conn(0x7f0d240c6150 0x7f0d240c8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:52.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.355+0000 7f0d2bc43640 1 --2- 172.21.15.44:0/1972832146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d240c5d70 0x7f0d24158ff0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:52.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.355+0000 7f0d2bc43640 1 -- 172.21.15.44:0/1972832146 >> 172.21.15.44:0/1972832146 conn(0x7f0d240ce700 msgr2=0x7f0d240ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:52.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.355+0000 7f0d2bc43640 1 -- 172.21.15.44:0/1972832146 shutdown_connections 2024-04-22T06:31:52.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.355+0000 7f0d2bc43640 1 -- 172.21.15.44:0/1972832146 wait complete. 2024-04-22T06:31:52.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.357+0000 7f0d2bc43640 1 Processor -- start 2024-04-22T06:31:52.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.357+0000 7f0d2bc43640 1 -- start start 2024-04-22T06:31:52.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.357+0000 7f0d2bc43640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d24070810 0x7f0d241ef420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:52.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.357+0000 7f0d2bc43640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d240c5d70 0x7f0d241578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:52.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.357+0000 7f0d2bc43640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d240c6150 0x7f0d2414cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:52.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.357+0000 7f0d2bc43640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0d240c8230 con 0x7f0d24070810 2024-04-22T06:31:52.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.357+0000 7f0d2bc43640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f0d240c7f50 con 0x7f0d240c5d70 2024-04-22T06:31:52.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.357+0000 7f0d2bc43640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f0d240738f0 con 0x7f0d240c6150 2024-04-22T06:31:52.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.357+0000 7f0d299df640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d24070810 0x7f0d241ef420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:52.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.357+0000 7f0d299df640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d24070810 0x7f0d241ef420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54950/0 (socket says 172.21.15.44:54950) 2024-04-22T06:31:52.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.357+0000 7f0d299df640 1 -- 172.21.15.44:0/2374553994 learned_addr learned my addr 172.21.15.44:0/2374553994 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:52.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.358+0000 7f0d291de640 1 --2- 172.21.15.44:0/2374553994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d240c5d70 0x7f0d241578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:52.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.358+0000 7f0d2a1e0640 1 --2- 172.21.15.44:0/2374553994 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d240c6150 0x7f0d2414cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:52.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.358+0000 7f0d299df640 1 -- 172.21.15.44:0/2374553994 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d240c6150 msgr2=0x7f0d2414cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:52.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.358+0000 7f0d299df640 1 --2- 172.21.15.44:0/2374553994 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d240c6150 0x7f0d2414cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:52.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.358+0000 7f0d299df640 1 -- 172.21.15.44:0/2374553994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d240c5d70 msgr2=0x7f0d241578d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:52.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.358+0000 7f0d299df640 1 --2- 172.21.15.44:0/2374553994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d240c5d70 0x7f0d241578d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:52.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.358+0000 7f0d299df640 1 -- 172.21.15.44:0/2374553994 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d24159dc0 con 0x7f0d24070810 2024-04-22T06:31:52.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.358+0000 7f0d299df640 1 --2- 172.21.15.44:0/2374553994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d24070810 0x7f0d241ef420 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0d20000c00 tx=0x7f0d20004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:52.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.358+0000 7f0d1a7fc640 1 -- 172.21.15.44:0/2374553994 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d20047070 con 0x7f0d24070810 2024-04-22T06:31:52.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.358+0000 7f0d1a7fc640 1 -- 172.21.15.44:0/2374553994 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d20037d30 con 0x7f0d24070810 2024-04-22T06:31:52.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.358+0000 7f0d2bc43640 1 -- 172.21.15.44:0/2374553994 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d24159ae0 con 0x7f0d24070810 2024-04-22T06:31:52.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.358+0000 7f0d1a7fc640 1 -- 172.21.15.44:0/2374553994 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d200426c0 con 0x7f0d24070810 2024-04-22T06:31:52.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.358+0000 7f0d2bc43640 1 -- 172.21.15.44:0/2374553994 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d2414a770 con 0x7f0d24070810 2024-04-22T06:31:52.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.359+0000 7f0d2bc43640 1 -- 172.21.15.44:0/2374553994 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d240762b0 con 0x7f0d24070810 2024-04-22T06:31:52.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.360+0000 7f0d1a7fc640 1 -- 172.21.15.44:0/2374553994 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0d2004a070 con 0x7f0d24070810 2024-04-22T06:31:52.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.360+0000 7f0d1a7fc640 1 --2- 172.21.15.44:0/2374553994 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f0d0806c920 0x7f0d0806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:52.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.360+0000 7f0d1a7fc640 1 -- 172.21.15.44:0/2374553994 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f0d200ae200 con 0x7f0d24070810 2024-04-22T06:31:52.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.360+0000 7f0d291de640 1 --2- 172.21.15.44:0/2374553994 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f0d0806c920 0x7f0d0806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:52.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.361+0000 7f0d291de640 1 --2- 172.21.15.44:0/2374553994 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f0d0806c920 0x7f0d0806ede0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0d24158c40 tx=0x7f0d14009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:52.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.365+0000 7f0d1a7fc640 1 -- 172.21.15.44:0/2374553994 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0d240762b0 con 0x7f0d24070810 2024-04-22T06:31:52.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.550+0000 7f0d2bc43640 1 -- 172.21.15.44:0/2374553994 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0d24069310 con 0x7f0d24070810 2024-04-22T06:31:52.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.550+0000 7f0d1a7fc640 1 -- 172.21.15.44:0/2374553994 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f0d20033030 con 0x7f0d24070810 2024-04-22T06:31:52.549 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-04-22T06:31:52.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.552+0000 7f0d2bc43640 1 -- 172.21.15.44:0/2374553994 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f0d0806c920 msgr2=0x7f0d0806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:52.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.552+0000 7f0d2bc43640 1 --2- 172.21.15.44:0/2374553994 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f0d0806c920 0x7f0d0806ede0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0d24158c40 tx=0x7f0d14009190 comp rx=0 tx=0).stop 2024-04-22T06:31:52.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.552+0000 7f0d2bc43640 1 -- 172.21.15.44:0/2374553994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d24070810 msgr2=0x7f0d241ef420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:52.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.552+0000 7f0d2bc43640 1 --2- 172.21.15.44:0/2374553994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d24070810 0x7f0d241ef420 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0d20000c00 tx=0x7f0d20004730 comp rx=0 tx=0).stop 2024-04-22T06:31:52.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.552+0000 7f0d2bc43640 1 -- 172.21.15.44:0/2374553994 shutdown_connections 2024-04-22T06:31:52.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.552+0000 7f0d2bc43640 1 --2- 172.21.15.44:0/2374553994 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f0d0806c920 0x7f0d0806ede0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:52.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.552+0000 7f0d2bc43640 1 --2- 172.21.15.44:0/2374553994 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d240c6150 0x7f0d2414cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:52.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.552+0000 7f0d2bc43640 1 --2- 172.21.15.44:0/2374553994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d240c5d70 0x7f0d241578d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:52.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.552+0000 7f0d2bc43640 1 --2- 172.21.15.44:0/2374553994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d24070810 0x7f0d241ef420 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:52.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.552+0000 7f0d2bc43640 1 -- 172.21.15.44:0/2374553994 >> 172.21.15.44:0/2374553994 conn(0x7f0d240ce700 msgr2=0x7f0d241522c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:52.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.552+0000 7f0d2bc43640 1 -- 172.21.15.44:0/2374553994 shutdown_connections 2024-04-22T06:31:52.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.552+0000 7f0d2bc43640 1 -- 172.21.15.44:0/2374553994 wait complete. 2024-04-22T06:31:52.562 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T06:31:52.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.666+0000 7f8be238f640 1 Processor -- start 2024-04-22T06:31:52.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.666+0000 7f8be238f640 1 -- start start 2024-04-22T06:31:52.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.666+0000 7f8be238f640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f8bdc06d470 0x7f8bdc0cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:52.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.666+0000 7f8be238f640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bdc15ccf0 con 0x7f8bdc154f00 2024-04-22T06:31:52.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.666+0000 7f8be238f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bdc0c5b30 con 0x7f8bdc1552e0 2024-04-22T06:31:52.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.666+0000 7f8be238f640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f8bdc0c6740 con 0x7f8bdc06d470 2024-04-22T06:31:52.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.667+0000 7f8be092c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f8bdc06d470 0x7f8bdc0cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:52.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.667+0000 7f8bdbfff640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f8bdc154f00 0x7f8bdc15c5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:38652/0 (socket says 172.21.15.44:38652) 2024-04-22T06:31:52.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.667+0000 7f8bdbfff640 1 -- 172.21.15.44:0/1527029652 learned_addr learned my addr 172.21.15.44:0/1527029652 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:52.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.667+0000 7f8be092c640 1 -- 172.21.15.44:0/1527029652 >> v1:172.21.15.70:6789/0 conn(0x7f8bdc1552e0 legacy=0x7f8bdc0d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:52.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.667+0000 7f8be092c640 1 -- 172.21.15.44:0/1527029652 >> v1:172.21.15.44:6789/0 conn(0x7f8bdc154f00 legacy=0x7f8bdc15c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:52.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.667+0000 7f8bdaffd640 1 -- 172.21.15.44:0/1527029652 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1142593070 0 0) 0x7f8bdc0c5b30 con 0x7f8bdc1552e0 2024-04-22T06:31:52.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.667+0000 7f8be092c640 1 -- 172.21.15.44:0/1527029652 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bdc0c5e10 con 0x7f8bdc06d470 2024-04-22T06:31:52.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.668+0000 7f8be092c640 1 --2- 172.21.15.44:0/1527029652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bdc06d470 0x7f8bdc0cb430 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4009d90 tx=0x7f8bd402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a6b88a8aa2ea8a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:52.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.668+0000 7f8bdaffd640 1 -- 172.21.15.44:0/1527029652 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd4039070 con 0x7f8bdc06d470 2024-04-22T06:31:52.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.668+0000 7f8bdaffd640 1 -- 172.21.15.44:0/1527029652 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bd402cc60 con 0x7f8bdc06d470 2024-04-22T06:31:52.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.668+0000 7f8bdaffd640 1 -- 172.21.15.44:0/1527029652 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd4034660 con 0x7f8bdc06d470 2024-04-22T06:31:52.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.668+0000 7f8be238f640 1 -- 172.21.15.44:0/1527029652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bdc06d470 msgr2=0x7f8bdc0cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:52.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.668+0000 7f8be238f640 1 --2- 172.21.15.44:0/1527029652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bdc06d470 0x7f8bdc0cb430 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4009d90 tx=0x7f8bd402beb0 comp rx=0 tx=0).stop 2024-04-22T06:31:52.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.668+0000 7f8be238f640 1 -- 172.21.15.44:0/1527029652 shutdown_connections 2024-04-22T06:31:52.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.668+0000 7f8be238f640 1 --2- 172.21.15.44:0/1527029652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bdc06d470 0x7f8bdc0cb430 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:52.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.668+0000 7f8be238f640 1 -- 172.21.15.44:0/1527029652 >> 172.21.15.44:0/1527029652 conn(0x7f8bdc0733c0 msgr2=0x7f8bdc0737d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:52.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.668+0000 7f8be238f640 1 -- 172.21.15.44:0/1527029652 shutdown_connections 2024-04-22T06:31:52.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.669+0000 7f8be238f640 1 -- 172.21.15.44:0/1527029652 wait complete. 2024-04-22T06:31:52.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.670+0000 7f8be238f640 1 Processor -- start 2024-04-22T06:31:52.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.670+0000 7f8be238f640 1 -- start start 2024-04-22T06:31:52.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.670+0000 7f8be238f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bdc06d470 0x7f8bdc14c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:52.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.670+0000 7f8be238f640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8bdc154f00 0x7f8bdc0b4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:52.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.670+0000 7f8be238f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bdc1552e0 0x7f8bdc0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:52.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.670+0000 7f8be238f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8bdc0c6bd0 con 0x7f8bdc06d470 2024-04-22T06:31:52.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.670+0000 7f8be238f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8bdc0c63d0 con 0x7f8bdc1552e0 2024-04-22T06:31:52.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.671+0000 7f8be238f640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8bdc0c6d50 con 0x7f8bdc154f00 2024-04-22T06:31:52.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.671+0000 7f8bdbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bdc06d470 0x7f8bdc14c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:52.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.671+0000 7f8bdbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bdc06d470 0x7f8bdc14c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54958/0 (socket says 172.21.15.44:54958) 2024-04-22T06:31:52.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.671+0000 7f8bdbfff640 1 -- 172.21.15.44:0/3580279451 learned_addr learned my addr 172.21.15.44:0/3580279451 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:52.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.671+0000 7f8bdb7fe640 1 --2- 172.21.15.44:0/3580279451 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8bdc154f00 0x7f8bdc0b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:52.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.671+0000 7f8be092c640 1 --2- 172.21.15.44:0/3580279451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bdc1552e0 0x7f8bdc0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:52.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.671+0000 7f8bdbfff640 1 -- 172.21.15.44:0/3580279451 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8bdc154f00 msgr2=0x7f8bdc0b4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:52.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.671+0000 7f8bdbfff640 1 --2- 172.21.15.44:0/3580279451 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8bdc154f00 0x7f8bdc0b4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:52.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.671+0000 7f8bdbfff640 1 -- 172.21.15.44:0/3580279451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bdc1552e0 msgr2=0x7f8bdc0b4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:52.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.671+0000 7f8bdbfff640 1 --2- 172.21.15.44:0/3580279451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bdc1552e0 0x7f8bdc0b4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:52.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.671+0000 7f8bdbfff640 1 -- 172.21.15.44:0/3580279451 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bdc1f0b90 con 0x7f8bdc06d470 2024-04-22T06:31:52.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.671+0000 7f8bdbfff640 1 --2- 172.21.15.44:0/3580279451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bdc06d470 0x7f8bdc14c750 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00c430 tx=0x7f8bcc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:52.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.671+0000 7f8bd8ff9640 1 -- 172.21.15.44:0/3580279451 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc017070 con 0x7f8bdc06d470 2024-04-22T06:31:52.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.671+0000 7f8bd8ff9640 1 -- 172.21.15.44:0/3580279451 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bcc005ce0 con 0x7f8bdc06d470 2024-04-22T06:31:52.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.671+0000 7f8be238f640 1 -- 172.21.15.44:0/3580279451 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bdc1f4280 con 0x7f8bdc06d470 2024-04-22T06:31:52.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.671+0000 7f8bd8ff9640 1 -- 172.21.15.44:0/3580279451 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc0072f0 con 0x7f8bdc06d470 2024-04-22T06:31:52.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.671+0000 7f8be238f640 1 -- 172.21.15.44:0/3580279451 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bdc1f10f0 con 0x7f8bdc06d470 2024-04-22T06:31:52.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.672+0000 7f8be238f640 1 -- 172.21.15.44:0/3580279451 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bdc15f0e0 con 0x7f8bdc06d470 2024-04-22T06:31:52.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.673+0000 7f8bd8ff9640 1 -- 172.21.15.44:0/3580279451 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8bcc01a070 con 0x7f8bdc06d470 2024-04-22T06:31:52.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.673+0000 7f8bd8ff9640 1 --2- 172.21.15.44:0/3580279451 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f8bc806c8f0 0x7f8bc806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:52.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.673+0000 7f8bd8ff9640 1 -- 172.21.15.44:0/3580279451 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f8bcc08b350 con 0x7f8bdc06d470 2024-04-22T06:31:52.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.673+0000 7f8bdb7fe640 1 --2- 172.21.15.44:0/3580279451 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f8bc806c8f0 0x7f8bc806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:52.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.674+0000 7f8bdb7fe640 1 --2- 172.21.15.44:0/3580279451 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f8bc806c8f0 0x7f8bc806edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc15eab0 tx=0x7f8bd0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:52.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.677+0000 7f8bd8ff9640 1 -- 172.21.15.44:0/3580279451 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8bdc15f0e0 con 0x7f8bdc06d470 2024-04-22T06:31:52.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.863+0000 7f8be238f640 1 -- 172.21.15.44:0/3580279451 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8bdc0b3170 con 0x7f8bdc06d470 2024-04-22T06:31:52.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.864+0000 7f8bd8ff9640 1 -- 172.21.15.44:0/3580279451 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f8bdc1f0b90 con 0x7f8bdc06d470 2024-04-22T06:31:52.862 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-04-22T06:31:52.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.866+0000 7f8be238f640 1 -- 172.21.15.44:0/3580279451 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f8bc806c8f0 msgr2=0x7f8bc806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:52.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.866+0000 7f8be238f640 1 --2- 172.21.15.44:0/3580279451 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f8bc806c8f0 0x7f8bc806edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc15eab0 tx=0x7f8bd0009040 comp rx=0 tx=0).stop 2024-04-22T06:31:52.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.866+0000 7f8be238f640 1 -- 172.21.15.44:0/3580279451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bdc06d470 msgr2=0x7f8bdc14c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:52.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.866+0000 7f8be238f640 1 --2- 172.21.15.44:0/3580279451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bdc06d470 0x7f8bdc14c750 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00c430 tx=0x7f8bcc00c8f0 comp rx=0 tx=0).stop 2024-04-22T06:31:52.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.866+0000 7f8be238f640 1 -- 172.21.15.44:0/3580279451 shutdown_connections 2024-04-22T06:31:52.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.866+0000 7f8be238f640 1 --2- 172.21.15.44:0/3580279451 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f8bc806c8f0 0x7f8bc806edb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:52.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.866+0000 7f8be238f640 1 --2- 172.21.15.44:0/3580279451 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8bdc154f00 0x7f8bdc0b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:52.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.866+0000 7f8be238f640 1 --2- 172.21.15.44:0/3580279451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bdc1552e0 0x7f8bdc0b4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:52.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.866+0000 7f8be238f640 1 --2- 172.21.15.44:0/3580279451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bdc06d470 0x7f8bdc14c750 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:52.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.866+0000 7f8be238f640 1 -- 172.21.15.44:0/3580279451 >> 172.21.15.44:0/3580279451 conn(0x7f8bdc0733c0 msgr2=0x7f8bdc14d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:52.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.866+0000 7f8be238f640 1 -- 172.21.15.44:0/3580279451 shutdown_connections 2024-04-22T06:31:52.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.866+0000 7f8be238f640 1 -- 172.21.15.44:0/3580279451 wait complete. 2024-04-22T06:31:52.875 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-22T06:31:52.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.979+0000 7f3c722a0640 1 Processor -- start 2024-04-22T06:31:52.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.980+0000 7f3c722a0640 1 -- start start 2024-04-22T06:31:52.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.980+0000 7f3c722a0640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3c6c074f20 0x7f3c6c0cab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:52.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.980+0000 7f3c722a0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3c6c075300 0x7f3c6c15f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:52.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.980+0000 7f3c722a0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c6c15f6d0 con 0x7f3c6c072130 2024-04-22T06:31:52.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.980+0000 7f3c722a0640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f3c6c0cb1d0 con 0x7f3c6c074f20 2024-04-22T06:31:52.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.980+0000 7f3c722a0640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f3c6c0744d0 con 0x7f3c6c075300 2024-04-22T06:31:52.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.980+0000 7f3c6bfff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3c6c074f20 0x7f3c6c0cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:52.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.981+0000 7f3c6bfff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3c6c074f20 0x7f3c6c0cab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54972/0 (socket says 172.21.15.44:54972) 2024-04-22T06:31:52.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.981+0000 7f3c6bfff640 1 -- 172.21.15.44:0/2636781506 learned_addr learned my addr 172.21.15.44:0/2636781506 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:52.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.981+0000 7f3c6b7fe640 1 --2- 172.21.15.44:0/2636781506 >> v2:172.21.15.70:3300/0 conn(0x7f3c6c075300 0x7f3c6c15f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:52.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.981+0000 7f3c6bfff640 1 -- 172.21.15.44:0/2636781506 >> v2:172.21.15.70:3300/0 conn(0x7f3c6c075300 msgr2=0x7f3c6c15f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:52.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.981+0000 7f3c6bfff640 1 --2- 172.21.15.44:0/2636781506 >> v2:172.21.15.70:3300/0 conn(0x7f3c6c075300 0x7f3c6c15f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:52.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.981+0000 7f3c6bfff640 1 -- 172.21.15.44:0/2636781506 >> v1:172.21.15.70:6789/0 conn(0x7f3c6c072130 legacy=0x7f3c6c154bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:52.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.981+0000 7f3c6bfff640 1 -- 172.21.15.44:0/2636781506 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c6c0c95e0 con 0x7f3c6c074f20 2024-04-22T06:31:52.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.981+0000 7f3c6affd640 1 -- 172.21.15.44:0/2636781506 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3714076943 0 0) 0x7f3c6c15f6d0 con 0x7f3c6c072130 2024-04-22T06:31:52.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.981+0000 7f3c6b7fe640 1 --2- 172.21.15.44:0/2636781506 >> v2:172.21.15.70:3300/0 conn(0x7f3c6c075300 0x7f3c6c15f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:31:52.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.982+0000 7f3c6bfff640 1 --2- 172.21.15.44:0/2636781506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c6c074f20 0x7f3c6c0cab40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c0c9200 tx=0x7f3c5402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3564c0f1fddee69 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:52.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.982+0000 7f3c6affd640 1 -- 172.21.15.44:0/2636781506 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5402f840 con 0x7f3c6c074f20 2024-04-22T06:31:52.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.982+0000 7f3c6affd640 1 -- 172.21.15.44:0/2636781506 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c5402f9e0 con 0x7f3c6c074f20 2024-04-22T06:31:52.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.982+0000 7f3c6affd640 1 -- 172.21.15.44:0/2636781506 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5402fdb0 con 0x7f3c6c074f20 2024-04-22T06:31:52.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.982+0000 7f3c722a0640 1 -- 172.21.15.44:0/2636781506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c6c074f20 msgr2=0x7f3c6c0cab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:52.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.982+0000 7f3c722a0640 1 --2- 172.21.15.44:0/2636781506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c6c074f20 0x7f3c6c0cab40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c0c9200 tx=0x7f3c5402ba20 comp rx=0 tx=0).stop 2024-04-22T06:31:52.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.982+0000 7f3c722a0640 1 -- 172.21.15.44:0/2636781506 shutdown_connections 2024-04-22T06:31:52.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.982+0000 7f3c722a0640 1 --2- 172.21.15.44:0/2636781506 >> v2:172.21.15.70:3300/0 conn(0x7f3c6c075300 0x7f3c6c15f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:52.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.982+0000 7f3c722a0640 1 --2- 172.21.15.44:0/2636781506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c6c074f20 0x7f3c6c0cab40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:52.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.982+0000 7f3c722a0640 1 -- 172.21.15.44:0/2636781506 >> 172.21.15.44:0/2636781506 conn(0x7f3c6c06b930 msgr2=0x7f3c6c06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:52.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.982+0000 7f3c722a0640 1 -- 172.21.15.44:0/2636781506 shutdown_connections 2024-04-22T06:31:52.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.982+0000 7f3c722a0640 1 -- 172.21.15.44:0/2636781506 wait complete. 2024-04-22T06:31:52.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.984+0000 7f3c722a0640 1 Processor -- start 2024-04-22T06:31:52.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.984+0000 7f3c722a0640 1 -- start start 2024-04-22T06:31:52.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.984+0000 7f3c722a0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c6c072130 0x7f3c6c146be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:52.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.984+0000 7f3c722a0640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3c6c074f20 0x7f3c6c0cbef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:52.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.984+0000 7f3c722a0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c6c075300 0x7f3c6c1437b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:52.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.984+0000 7f3c722a0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3c6c0c9b60 con 0x7f3c6c072130 2024-04-22T06:31:52.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.984+0000 7f3c722a0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f3c6c0ca1b0 con 0x7f3c6c075300 2024-04-22T06:31:52.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.984+0000 7f3c722a0640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3c6c0c9e40 con 0x7f3c6c074f20 2024-04-22T06:31:52.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.985+0000 7f3c6bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c6c072130 0x7f3c6c146be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:52.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.985+0000 7f3c6bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c6c072130 0x7f3c6c146be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54980/0 (socket says 172.21.15.44:54980) 2024-04-22T06:31:52.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.985+0000 7f3c6bfff640 1 -- 172.21.15.44:0/1858438817 learned_addr learned my addr 172.21.15.44:0/1858438817 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:52.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.985+0000 7f3c7083d640 1 --2- 172.21.15.44:0/1858438817 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c6c075300 0x7f3c6c1437b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:52.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.985+0000 7f3c6b7fe640 1 --2- 172.21.15.44:0/1858438817 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3c6c074f20 0x7f3c6c0cbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:52.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.985+0000 7f3c6bfff640 1 -- 172.21.15.44:0/1858438817 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3c6c074f20 msgr2=0x7f3c6c0cbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:52.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.985+0000 7f3c6bfff640 1 --2- 172.21.15.44:0/1858438817 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3c6c074f20 0x7f3c6c0cbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:52.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.985+0000 7f3c6bfff640 1 -- 172.21.15.44:0/1858438817 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c6c075300 msgr2=0x7f3c6c1437b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:52.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.985+0000 7f3c6bfff640 1 --2- 172.21.15.44:0/1858438817 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c6c075300 0x7f3c6c1437b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:52.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.985+0000 7f3c6bfff640 1 -- 172.21.15.44:0/1858438817 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c6c159f80 con 0x7f3c6c072130 2024-04-22T06:31:52.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.985+0000 7f3c6bfff640 1 --2- 172.21.15.44:0/1858438817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c6c072130 0x7f3c6c146be0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3c540090b0 tx=0x7f3c54004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:52.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.985+0000 7f3c68ff9640 1 -- 172.21.15.44:0/1858438817 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c54047070 con 0x7f3c6c072130 2024-04-22T06:31:52.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.985+0000 7f3c68ff9640 1 -- 172.21.15.44:0/1858438817 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c54037bc0 con 0x7f3c6c072130 2024-04-22T06:31:52.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.985+0000 7f3c722a0640 1 -- 172.21.15.44:0/1858438817 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c6c159140 con 0x7f3c6c072130 2024-04-22T06:31:52.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.985+0000 7f3c68ff9640 1 -- 172.21.15.44:0/1858438817 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c54004030 con 0x7f3c6c072130 2024-04-22T06:31:52.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.985+0000 7f3c722a0640 1 -- 172.21.15.44:0/1858438817 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c6c0ce5b0 con 0x7f3c6c072130 2024-04-22T06:31:52.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.987+0000 7f3c722a0640 1 -- 172.21.15.44:0/1858438817 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c6c069310 con 0x7f3c6c072130 2024-04-22T06:31:52.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.987+0000 7f3c68ff9640 1 -- 172.21.15.44:0/1858438817 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3c5404a070 con 0x7f3c6c072130 2024-04-22T06:31:52.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.987+0000 7f3c68ff9640 1 --2- 172.21.15.44:0/1858438817 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f3c4c06c920 0x7f3c4c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:52.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.988+0000 7f3c68ff9640 1 -- 172.21.15.44:0/1858438817 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f3c540adfe0 con 0x7f3c6c072130 2024-04-22T06:31:52.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.988+0000 7f3c6b7fe640 1 --2- 172.21.15.44:0/1858438817 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f3c4c06c920 0x7f3c4c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:52.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.988+0000 7f3c6b7fe640 1 --2- 172.21.15.44:0/1858438817 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f3c4c06c920 0x7f3c4c06ede0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c009560 tx=0x7f3c5c009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:52.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:52.992+0000 7f3c68ff9640 1 -- 172.21.15.44:0/1858438817 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3c54033030 con 0x7f3c6c072130 2024-04-22T06:31:53.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.179+0000 7f3c722a0640 1 -- 172.21.15.44:0/1858438817 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f3c6c0c8840 con 0x7f3c6c072130 2024-04-22T06:31:53.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.180+0000 7f3c68ff9640 1 -- 172.21.15.44:0/1858438817 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f3c5407b930 con 0x7f3c6c072130 2024-04-22T06:31:53.179 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-04-22T06:31:53.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.182+0000 7f3c722a0640 1 -- 172.21.15.44:0/1858438817 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f3c4c06c920 msgr2=0x7f3c4c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:53.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.182+0000 7f3c722a0640 1 --2- 172.21.15.44:0/1858438817 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f3c4c06c920 0x7f3c4c06ede0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c009560 tx=0x7f3c5c009190 comp rx=0 tx=0).stop 2024-04-22T06:31:53.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.182+0000 7f3c722a0640 1 -- 172.21.15.44:0/1858438817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c6c072130 msgr2=0x7f3c6c146be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:53.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.182+0000 7f3c722a0640 1 --2- 172.21.15.44:0/1858438817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c6c072130 0x7f3c6c146be0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3c540090b0 tx=0x7f3c54004730 comp rx=0 tx=0).stop 2024-04-22T06:31:53.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.182+0000 7f3c722a0640 1 -- 172.21.15.44:0/1858438817 shutdown_connections 2024-04-22T06:31:53.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.182+0000 7f3c722a0640 1 --2- 172.21.15.44:0/1858438817 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f3c4c06c920 0x7f3c4c06ede0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:53.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.182+0000 7f3c722a0640 1 --2- 172.21.15.44:0/1858438817 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3c6c074f20 0x7f3c6c0cbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:53.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.182+0000 7f3c722a0640 1 --2- 172.21.15.44:0/1858438817 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c6c075300 0x7f3c6c1437b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:53.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.182+0000 7f3c722a0640 1 --2- 172.21.15.44:0/1858438817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c6c072130 0x7f3c6c146be0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:53.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.182+0000 7f3c722a0640 1 -- 172.21.15.44:0/1858438817 >> 172.21.15.44:0/1858438817 conn(0x7f3c6c06b930 msgr2=0x7f3c6c0725e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:53.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.182+0000 7f3c722a0640 1 -- 172.21.15.44:0/1858438817 shutdown_connections 2024-04-22T06:31:53.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.182+0000 7f3c722a0640 1 -- 172.21.15.44:0/1858438817 wait complete. 2024-04-22T06:31:53.192 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T06:31:53.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.293+0000 7f8fa4a81640 1 Processor -- start 2024-04-22T06:31:53.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.293+0000 7f8fa4a81640 1 -- start start 2024-04-22T06:31:53.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.294+0000 7f8fa4a81640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f8fa00ce960 0x7f8fa0074220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:53.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.294+0000 7f8fa4a81640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8fa00cabe0 0x7f8fa0152c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:53.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.294+0000 7f8fa4a81640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fa0073a10 con 0x7f8fa00ced40 2024-04-22T06:31:53.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.294+0000 7f8fa4a81640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f8fa00cb050 con 0x7f8fa00ce960 2024-04-22T06:31:53.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.294+0000 7f8fa4a81640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8fa0073cb0 con 0x7f8fa00cabe0 2024-04-22T06:31:53.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.294+0000 7f8f9e59c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f8fa00ce960 0x7f8fa0074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:53.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.294+0000 7f8f9e59c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f8fa00ce960 0x7f8fa0074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54984/0 (socket says 172.21.15.44:54984) 2024-04-22T06:31:53.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.294+0000 7f8f9e59c640 1 -- 172.21.15.44:0/1064879370 learned_addr learned my addr 172.21.15.44:0/1064879370 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:53.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.294+0000 7f8f9ed9d640 1 --2- 172.21.15.44:0/1064879370 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8fa00cabe0 0x7f8fa0152c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:53.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.295+0000 7f8f9e59c640 1 -- 172.21.15.44:0/1064879370 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8fa00cabe0 msgr2=0x7f8fa0152c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:53.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.295+0000 7f8f9e59c640 1 --2- 172.21.15.44:0/1064879370 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8fa00cabe0 0x7f8fa0152c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:53.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.295+0000 7f8f9d59a640 1 -- 172.21.15.44:0/1064879370 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 192840803 0 0) 0x7f8fa0073a10 con 0x7f8fa00ced40 2024-04-22T06:31:53.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.295+0000 7f8f9e59c640 1 -- 172.21.15.44:0/1064879370 >> v1:172.21.15.70:6789/0 conn(0x7f8fa00ced40 legacy=0x7f8fa00ca130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:53.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.295+0000 7f8f9e59c640 1 -- 172.21.15.44:0/1064879370 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fa0073f90 con 0x7f8fa00ce960 2024-04-22T06:31:53.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.295+0000 7f8f9e59c640 1 --2- 172.21.15.44:0/1064879370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8fa00ce960 0x7f8fa0074220 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8f98009870 tx=0x7f8f9802ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=824c1b0d1bd9c9f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:53.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.296+0000 7f8f9d59a640 1 -- 172.21.15.44:0/1064879370 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9802f840 con 0x7f8fa00ce960 2024-04-22T06:31:53.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.296+0000 7f8f9d59a640 1 -- 172.21.15.44:0/1064879370 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f9802f9e0 con 0x7f8fa00ce960 2024-04-22T06:31:53.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.296+0000 7f8f9d59a640 1 -- 172.21.15.44:0/1064879370 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9802fdb0 con 0x7f8fa00ce960 2024-04-22T06:31:53.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.296+0000 7f8fa4a81640 1 -- 172.21.15.44:0/1064879370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8fa00ce960 msgr2=0x7f8fa0074220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:53.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.296+0000 7f8fa4a81640 1 --2- 172.21.15.44:0/1064879370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8fa00ce960 0x7f8fa0074220 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8f98009870 tx=0x7f8f9802ba20 comp rx=0 tx=0).stop 2024-04-22T06:31:53.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.296+0000 7f8fa4a81640 1 -- 172.21.15.44:0/1064879370 shutdown_connections 2024-04-22T06:31:53.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.296+0000 7f8fa4a81640 1 --2- 172.21.15.44:0/1064879370 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8fa00cabe0 0x7f8fa0152c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:53.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.296+0000 7f8fa4a81640 1 --2- 172.21.15.44:0/1064879370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8fa00ce960 0x7f8fa0074220 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:53.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.296+0000 7f8fa4a81640 1 -- 172.21.15.44:0/1064879370 >> 172.21.15.44:0/1064879370 conn(0x7f8fa0074cb0 msgr2=0x7f8fa00750c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:53.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.296+0000 7f8fa4a81640 1 -- 172.21.15.44:0/1064879370 shutdown_connections 2024-04-22T06:31:53.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.296+0000 7f8fa4a81640 1 -- 172.21.15.44:0/1064879370 wait complete. 2024-04-22T06:31:53.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.297+0000 7f8fa4a81640 1 Processor -- start 2024-04-22T06:31:53.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.297+0000 7f8fa4a81640 1 -- start start 2024-04-22T06:31:53.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.298+0000 7f8fa4a81640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8fa00cabe0 0x7f8fa00c5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:53.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.298+0000 7f8fa4a81640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8fa00ce960 0x7f8fa013f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:53.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.298+0000 7f8fa4a81640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fa00ced40 0x7f8fa01425a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:53.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.298+0000 7f8fa4a81640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa00c97c0 con 0x7f8fa00ce960 2024-04-22T06:31:53.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.298+0000 7f8fa4a81640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa00c94e0 con 0x7f8fa00ced40 2024-04-22T06:31:53.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.298+0000 7f8fa4a81640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8fa00c9be0 con 0x7f8fa00cabe0 2024-04-22T06:31:53.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.298+0000 7f8f9dd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8fa00ce960 0x7f8fa013f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:53.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.298+0000 7f8f9e59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8fa00cabe0 0x7f8fa00c5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:53.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.298+0000 7f8f9dd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8fa00ce960 0x7f8fa013f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54992/0 (socket says 172.21.15.44:54992) 2024-04-22T06:31:53.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.298+0000 7f8f9dd9b640 1 -- 172.21.15.44:0/3676682450 learned_addr learned my addr 172.21.15.44:0/3676682450 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:53.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.298+0000 7f8f9ed9d640 1 --2- 172.21.15.44:0/3676682450 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fa00ced40 0x7f8fa01425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:53.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.299+0000 7f8f9dd9b640 1 -- 172.21.15.44:0/3676682450 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8fa00cabe0 msgr2=0x7f8fa00c5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:53.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.299+0000 7f8f9dd9b640 1 --2- 172.21.15.44:0/3676682450 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8fa00cabe0 0x7f8fa00c5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:53.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.299+0000 7f8f9dd9b640 1 -- 172.21.15.44:0/3676682450 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fa00ced40 msgr2=0x7f8fa01425a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:53.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.299+0000 7f8f9dd9b640 1 --2- 172.21.15.44:0/3676682450 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fa00ced40 0x7f8fa01425a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:53.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.299+0000 7f8f9dd9b640 1 -- 172.21.15.44:0/3676682450 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fa0141690 con 0x7f8fa00ce960 2024-04-22T06:31:53.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.299+0000 7f8f9e59c640 1 --2- 172.21.15.44:0/3676682450 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8fa00cabe0 0x7f8fa00c5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:31:53.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.299+0000 7f8f9ed9d640 1 --2- 172.21.15.44:0/3676682450 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fa00ced40 0x7f8fa01425a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:31:53.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.299+0000 7f8f9dd9b640 1 --2- 172.21.15.44:0/3676682450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8fa00ce960 0x7f8fa013f660 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800c3e0 tx=0x7f8f8800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:53.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.299+0000 7f8f96ffd640 1 -- 172.21.15.44:0/3676682450 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f88017070 con 0x7f8fa00ce960 2024-04-22T06:31:53.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.299+0000 7f8f96ffd640 1 -- 172.21.15.44:0/3676682450 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f880035f0 con 0x7f8fa00ce960 2024-04-22T06:31:53.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.299+0000 7f8fa4a81640 1 -- 172.21.15.44:0/3676682450 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fa0141900 con 0x7f8fa00ce960 2024-04-22T06:31:53.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.299+0000 7f8f96ffd640 1 -- 172.21.15.44:0/3676682450 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f88003a00 con 0x7f8fa00ce960 2024-04-22T06:31:53.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.299+0000 7f8fa4a81640 1 -- 172.21.15.44:0/3676682450 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fa0144390 con 0x7f8fa00ce960 2024-04-22T06:31:53.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.300+0000 7f8fa4a81640 1 -- 172.21.15.44:0/3676682450 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fa013eb80 con 0x7f8fa00ce960 2024-04-22T06:31:53.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.300+0000 7f8f96ffd640 1 -- 172.21.15.44:0/3676682450 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8f8801a070 con 0x7f8fa00ce960 2024-04-22T06:31:53.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.301+0000 7f8f96ffd640 1 --2- 172.21.15.44:0/3676682450 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f8f8006c920 0x7f8f8006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:53.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.301+0000 7f8f96ffd640 1 -- 172.21.15.44:0/3676682450 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f8f8808b350 con 0x7f8fa00ce960 2024-04-22T06:31:53.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.301+0000 7f8f9e59c640 1 --2- 172.21.15.44:0/3676682450 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f8f8006c920 0x7f8f8006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:53.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.301+0000 7f8f9e59c640 1 --2- 172.21.15.44:0/3676682450 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f8f8006c920 0x7f8f8006ede0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8f98036000 tx=0x7f8f98008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:53.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.305+0000 7f8f96ffd640 1 -- 172.21.15.44:0/3676682450 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8fa013eb80 con 0x7f8fa00ce960 2024-04-22T06:31:53.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.533+0000 7f8fa4a81640 1 -- 172.21.15.44:0/3676682450 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8fa0069310 con 0x7f8fa00ce960 2024-04-22T06:31:53.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.534+0000 7f8f96ffd640 1 -- 172.21.15.44:0/3676682450 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6901 (secure 0 0 0) 0x7f8f88058ca0 con 0x7f8fa00ce960 2024-04-22T06:31:53.533 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:31:53.533 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/1779271860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":1779271860},{"type":"v1","addr":"172.21.15.70:6836","nonce":1779271860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T06:31:06.356312+0000","modified":"2024-04-22T06:31:50.151090+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4308,"mds_1":4305,"mds_2":4293,"mds_3":4290,"mds_4":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":4,"incarnation":17,"state":"up:creating","state_seq":1,"addr":"172.21.15.44:6834/1497302315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1497302315},{"type":"v1","addr":"172.21.15.44:6834","nonce":1497302315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4290":{"gid":4290,"name":"a","rank":3,"incarnation":15,"state":"up:active","state_seq":14,"addr":"172.21.15.44:6835/2383621014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2383621014},{"type":"v1","addr":"172.21.15.44:6835","nonce":2383621014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"e","rank":2,"incarnation":13,"state":"up:active","state_seq":12,"addr":"172.21.15.44:6837/915960587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":915960587},{"type":"v1","addr":"172.21.15.44:6837","nonce":915960587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4305":{"gid":4305,"name":"b","rank":1,"incarnation":9,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6838/924955785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":924955785},{"type":"v1","addr":"172.21.15.70:6838","nonce":924955785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4308":{"gid":4308,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.70:6839/2915150040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6837","nonce":2915150040},{"type":"v1","addr":"172.21.15.70:6839","nonce":2915150040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4308,"qdb_cluster":[4308,4305,4293,4290]},"id":1}]} 2024-04-22T06:31:53.533 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 17 2024-04-22T06:31:53.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.535+0000 7f8fa4a81640 1 -- 172.21.15.44:0/3676682450 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f8f8006c920 msgr2=0x7f8f8006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:53.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.535+0000 7f8fa4a81640 1 --2- 172.21.15.44:0/3676682450 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f8f8006c920 0x7f8f8006ede0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8f98036000 tx=0x7f8f98008030 comp rx=0 tx=0).stop 2024-04-22T06:31:53.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.535+0000 7f8fa4a81640 1 -- 172.21.15.44:0/3676682450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8fa00ce960 msgr2=0x7f8fa013f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:53.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.535+0000 7f8fa4a81640 1 --2- 172.21.15.44:0/3676682450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8fa00ce960 0x7f8fa013f660 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800c3e0 tx=0x7f8f8800c8a0 comp rx=0 tx=0).stop 2024-04-22T06:31:53.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.536+0000 7f8fa4a81640 1 -- 172.21.15.44:0/3676682450 shutdown_connections 2024-04-22T06:31:53.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.536+0000 7f8fa4a81640 1 --2- 172.21.15.44:0/3676682450 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f8f8006c920 0x7f8f8006ede0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:53.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.536+0000 7f8fa4a81640 1 --2- 172.21.15.44:0/3676682450 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fa00ced40 0x7f8fa01425a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:53.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.536+0000 7f8fa4a81640 1 --2- 172.21.15.44:0/3676682450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8fa00ce960 0x7f8fa013f660 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:53.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.536+0000 7f8fa4a81640 1 --2- 172.21.15.44:0/3676682450 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8fa00cabe0 0x7f8fa00c5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:53.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.536+0000 7f8fa4a81640 1 -- 172.21.15.44:0/3676682450 >> 172.21.15.44:0/3676682450 conn(0x7f8fa0074cb0 msgr2=0x7f8fa00c7ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:53.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.536+0000 7f8fa4a81640 1 -- 172.21.15.44:0/3676682450 shutdown_connections 2024-04-22T06:31:53.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.536+0000 7f8fa4a81640 1 -- 172.21.15.44:0/3676682450 wait complete. 2024-04-22T06:31:53.545 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T06:31:53.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.647+0000 7fd0c5ac1640 1 Processor -- start 2024-04-22T06:31:53.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.647+0000 7fd0c5ac1640 1 -- start start 2024-04-22T06:31:53.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.648+0000 7fd0c5ac1640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd0c01530e0 0x7fd0c0071460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:53.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.648+0000 7fd0c5ac1640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0c00c9c50 0x7fd0c00ca060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:53.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.648+0000 7fd0c5ac1640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0c015c680 con 0x7fd0c01598e0 2024-04-22T06:31:53.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.648+0000 7fd0c5ac1640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fd0c015c450 con 0x7fd0c01530e0 2024-04-22T06:31:53.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.648+0000 7fd0c5ac1640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd0c00711e0 con 0x7fd0c00c9c50 2024-04-22T06:31:53.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.648+0000 7fd0bf7fe640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd0c01530e0 0x7fd0c0071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:53.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.648+0000 7fd0bffff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0c00c9c50 0x7fd0c00ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:53.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.648+0000 7fd0bf7fe640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd0c01530e0 0x7fd0c0071460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54996/0 (socket says 172.21.15.44:54996) 2024-04-22T06:31:53.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.648+0000 7fd0beffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fd0c01598e0 0x7fd0c00c5be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.44:36442/0 (socket says 172.21.15.44:36442) 2024-04-22T06:31:53.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.648+0000 7fd0bf7fe640 1 -- 172.21.15.44:0/1813446888 learned_addr learned my addr 172.21.15.44:0/1813446888 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:53.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.648+0000 7fd0be7fc640 1 -- 172.21.15.44:0/1813446888 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2923124939 0 0) 0x7fd0c015c680 con 0x7fd0c01598e0 2024-04-22T06:31:53.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.648+0000 7fd0be7fc640 1 -- 172.21.15.44:0/1813446888 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd0a8003520 con 0x7fd0c01598e0 2024-04-22T06:31:53.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.648+0000 7fd0bf7fe640 1 -- 172.21.15.44:0/1813446888 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0c00c9c50 msgr2=0x7fd0c00ca060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:53.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.648+0000 7fd0bf7fe640 1 --2- 172.21.15.44:0/1813446888 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0c00c9c50 0x7fd0c00ca060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:53.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.649+0000 7fd0bf7fe640 1 -- 172.21.15.44:0/1813446888 >> v1:172.21.15.70:6789/0 conn(0x7fd0c01598e0 legacy=0x7fd0c00c5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:53.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.649+0000 7fd0bf7fe640 1 -- 172.21.15.44:0/1813446888 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0c015c960 con 0x7fd0c01530e0 2024-04-22T06:31:53.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.649+0000 7fd0bf7fe640 1 --2- 172.21.15.44:0/1813446888 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0c01530e0 0x7fd0c0071460 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0009e80 tx=0x7fd0b002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7469bc59c1f43fac server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:53.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.649+0000 7fd0be7fc640 1 -- 172.21.15.44:0/1813446888 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b002caf0 con 0x7fd0c01530e0 2024-04-22T06:31:53.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.649+0000 7fd0be7fc640 1 -- 172.21.15.44:0/1813446888 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0b002cc90 con 0x7fd0c01530e0 2024-04-22T06:31:53.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.650+0000 7fd0be7fc640 1 -- 172.21.15.44:0/1813446888 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b0034610 con 0x7fd0c01530e0 2024-04-22T06:31:53.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.650+0000 7fd0c5ac1640 1 -- 172.21.15.44:0/1813446888 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0c01530e0 msgr2=0x7fd0c0071460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:53.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.650+0000 7fd0c5ac1640 1 --2- 172.21.15.44:0/1813446888 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0c01530e0 0x7fd0c0071460 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0009e80 tx=0x7fd0b002bf50 comp rx=0 tx=0).stop 2024-04-22T06:31:53.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.650+0000 7fd0c5ac1640 1 -- 172.21.15.44:0/1813446888 shutdown_connections 2024-04-22T06:31:53.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.650+0000 7fd0c5ac1640 1 --2- 172.21.15.44:0/1813446888 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0c00c9c50 0x7fd0c00ca060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:53.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.650+0000 7fd0c5ac1640 1 --2- 172.21.15.44:0/1813446888 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0c01530e0 0x7fd0c0071460 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:53.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.650+0000 7fd0c5ac1640 1 -- 172.21.15.44:0/1813446888 >> 172.21.15.44:0/1813446888 conn(0x7fd0c0074cb0 msgr2=0x7fd0c00750c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:53.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.650+0000 7fd0c5ac1640 1 -- 172.21.15.44:0/1813446888 shutdown_connections 2024-04-22T06:31:53.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.650+0000 7fd0c5ac1640 1 -- 172.21.15.44:0/1813446888 wait complete. 2024-04-22T06:31:53.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.651+0000 7fd0c5ac1640 1 Processor -- start 2024-04-22T06:31:53.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.651+0000 7fd0c5ac1640 1 -- start start 2024-04-22T06:31:53.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.652+0000 7fd0c5ac1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0c00c9c50 0x7fd0c01520b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:53.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.652+0000 7fd0c5ac1640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0c01530e0 0x7fd0c0145090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:53.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.652+0000 7fd0c5ac1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0c01598e0 0x7fd0c01481a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:53.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.652+0000 7fd0c5ac1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c015cf20 con 0x7fd0c00c9c50 2024-04-22T06:31:53.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.652+0000 7fd0c5ac1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c00c97c0 con 0x7fd0c01598e0 2024-04-22T06:31:53.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.652+0000 7fd0c5ac1640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd0c015d200 con 0x7fd0c01530e0 2024-04-22T06:31:53.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.652+0000 7fd0bf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0c00c9c50 0x7fd0c01520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:53.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.652+0000 7fd0bf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0c00c9c50 0x7fd0c01520b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55004/0 (socket says 172.21.15.44:55004) 2024-04-22T06:31:53.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.652+0000 7fd0bf7fe640 1 -- 172.21.15.44:0/3820720859 learned_addr learned my addr 172.21.15.44:0/3820720859 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:53.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.652+0000 7fd0beffd640 1 --2- 172.21.15.44:0/3820720859 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0c01530e0 0x7fd0c0145090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:53.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.652+0000 7fd0bffff640 1 --2- 172.21.15.44:0/3820720859 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0c01598e0 0x7fd0c01481a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:53.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.652+0000 7fd0bf7fe640 1 -- 172.21.15.44:0/3820720859 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0c01530e0 msgr2=0x7fd0c0145090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:53.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.652+0000 7fd0bf7fe640 1 --2- 172.21.15.44:0/3820720859 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0c01530e0 0x7fd0c0145090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:53.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.652+0000 7fd0bf7fe640 1 -- 172.21.15.44:0/3820720859 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0c01598e0 msgr2=0x7fd0c01481a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:53.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.652+0000 7fd0bf7fe640 1 --2- 172.21.15.44:0/3820720859 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0c01598e0 0x7fd0c01481a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:53.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.652+0000 7fd0bf7fe640 1 -- 172.21.15.44:0/3820720859 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0c015c1f0 con 0x7fd0c00c9c50 2024-04-22T06:31:53.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.653+0000 7fd0bf7fe640 1 --2- 172.21.15.44:0/3820720859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0c00c9c50 0x7fd0c01520b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0036040 tx=0x7fd0b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:53.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.653+0000 7fd0a7fff640 1 -- 172.21.15.44:0/3820720859 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b0043020 con 0x7fd0c00c9c50 2024-04-22T06:31:53.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.653+0000 7fd0a7fff640 1 -- 172.21.15.44:0/3820720859 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0b003e450 con 0x7fd0c00c9c50 2024-04-22T06:31:53.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.653+0000 7fd0c5ac1640 1 -- 172.21.15.44:0/3820720859 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0c015bf10 con 0x7fd0c00c9c50 2024-04-22T06:31:53.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.653+0000 7fd0a7fff640 1 -- 172.21.15.44:0/3820720859 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b0009510 con 0x7fd0c00c9c50 2024-04-22T06:31:53.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.653+0000 7fd0c5ac1640 1 -- 172.21.15.44:0/3820720859 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0c0143e60 con 0x7fd0c00c9c50 2024-04-22T06:31:53.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.654+0000 7fd0c5ac1640 1 -- 172.21.15.44:0/3820720859 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0c0150840 con 0x7fd0c00c9c50 2024-04-22T06:31:53.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.654+0000 7fd0a7fff640 1 -- 172.21.15.44:0/3820720859 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd0b0046020 con 0x7fd0c00c9c50 2024-04-22T06:31:53.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.655+0000 7fd0a7fff640 1 --2- 172.21.15.44:0/3820720859 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd0ac06c8f0 0x7fd0ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:53.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.655+0000 7fd0a7fff640 1 -- 172.21.15.44:0/3820720859 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fd0b00ae470 con 0x7fd0c00c9c50 2024-04-22T06:31:53.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.655+0000 7fd0beffd640 1 --2- 172.21.15.44:0/3820720859 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd0ac06c8f0 0x7fd0ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:53.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.655+0000 7fd0beffd640 1 --2- 172.21.15.44:0/3820720859 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd0ac06c8f0 0x7fd0ac06edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd0c00cf7e0 tx=0x7fd0b4007cf0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:53.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.659+0000 7fd0a7fff640 1 -- 172.21.15.44:0/3820720859 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd0c0150840 con 0x7fd0c00c9c50 2024-04-22T06:31:53.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.845+0000 7fd0c5ac1640 1 -- 172.21.15.44:0/3820720859 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd0c0068cc0 con 0x7fd0c00c9c50 2024-04-22T06:31:53.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.846+0000 7fd0a7fff640 1 -- 172.21.15.44:0/3820720859 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18794 (secure 0 0 0) 0x7fd0b004d030 con 0x7fd0c00c9c50 2024-04-22T06:31:53.845 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:31:53.845 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":27,"fsid":"9b0a34ca-403d-4513-8a66-d3749ba559aa","created":"2024-04-22T06:30:42.334581+0000","modified":"2024-04-22T06:31:48.138913+0000","last_up_change":"2024-04-22T06:30:52.942257+0000","last_in_change":"2024-04-22T06:30:44.467727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T06:30:54.560256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T06:31:04.612387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T06:31:05.312124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-22T06:31:09.025550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b4a6692-53c9-472d-b2f3-1dd29a6679ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6817","nonce":2220268493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6819","nonce":2220268493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6823","nonce":2220268493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6821","nonce":2220268493}]},"public_addr":"172.21.15.44:6817/2220268493","cluster_addr":"172.21.15.44:6819/2220268493","heartbeat_back_addr":"172.21.15.44:6823/2220268493","heartbeat_front_addr":"172.21.15.44:6821/2220268493","state":["exists","up"]},{"osd":1,"uuid":"70a6370a-7c0a-4d64-a0e4-b2e726b4f291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6809","nonce":1278678977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6811","nonce":1278678977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6815","nonce":1278678977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6813","nonce":1278678977}]},"public_addr":"172.21.15.44:6809/1278678977","cluster_addr":"172.21.15.44:6811/1278678977","heartbeat_back_addr":"172.21.15.44:6815/1278678977","heartbeat_front_addr":"172.21.15.44:6813/1278678977","state":["exists","up"]},{"osd":2,"uuid":"f2e51a62-ca91-470d-bd5a-5710c61deee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6801","nonce":2544087903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6803","nonce":2544087903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6807","nonce":2544087903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6805","nonce":2544087903}]},"public_addr":"172.21.15.44:6801/2544087903","cluster_addr":"172.21.15.44:6803/2544087903","heartbeat_back_addr":"172.21.15.44:6807/2544087903","heartbeat_front_addr":"172.21.15.44:6805/2544087903","state":["exists","up"]},{"osd":3,"uuid":"bc514203-9af9-4f1f-b18e-765bdd5de6e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6825","nonce":1692928109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6827","nonce":1692928109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6831","nonce":1692928109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6829","nonce":1692928109}]},"public_addr":"172.21.15.44:6825/1692928109","cluster_addr":"172.21.15.44:6827/1692928109","heartbeat_back_addr":"172.21.15.44:6831/1692928109","heartbeat_front_addr":"172.21.15.44:6829/1692928109","state":["exists","up"]},{"osd":4,"uuid":"357c439a-6e08-4c19-ac18-b68e6c890497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6820","nonce":3802577427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6824","nonce":3802577427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6831","nonce":3802577427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6828","nonce":3802577427}]},"public_addr":"172.21.15.70:6820/3802577427","cluster_addr":"172.21.15.70:6824/3802577427","heartbeat_back_addr":"172.21.15.70:6831/3802577427","heartbeat_front_addr":"172.21.15.70:6828/3802577427","state":["exists","up"]},{"osd":5,"uuid":"952fb88f-d56e-4305-bfa8-d32674332dfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6801","nonce":664757345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6803","nonce":664757345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6807","nonce":664757345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6805","nonce":664757345}]},"public_addr":"172.21.15.70:6801/664757345","cluster_addr":"172.21.15.70:6803/664757345","heartbeat_back_addr":"172.21.15.70:6807/664757345","heartbeat_front_addr":"172.21.15.70:6805/664757345","state":["exists","up"]},{"osd":6,"uuid":"74399b3d-fac2-45ab-9172-2cd95b02450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6817","nonce":1623253482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6821","nonce":1623253482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6827","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6829","nonce":1623253482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6825","nonce":1623253482}]},"public_addr":"172.21.15.70:6817/1623253482","cluster_addr":"172.21.15.70:6821/1623253482","heartbeat_back_addr":"172.21.15.70:6829/1623253482","heartbeat_front_addr":"172.21.15.70:6825/1623253482","state":["exists","up"]},{"osd":7,"uuid":"8f35508d-6fcc-43ff-8ee7-0498a7ab3b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6809","nonce":1551180300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6811","nonce":1551180300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6815","nonce":1551180300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6813","nonce":1551180300}]},"public_addr":"172.21.15.70:6809/1551180300","cluster_addr":"172.21.15.70:6811/1551180300","heartbeat_back_addr":"172.21.15.70:6815/1551180300","heartbeat_front_addr":"172.21.15.70:6813/1551180300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.612502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.653756+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.521169+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.649955+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.640231+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.686638+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.642199+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.700360+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T06:31:53.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.848+0000 7fd0c5ac1640 1 -- 172.21.15.44:0/3820720859 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd0ac06c8f0 msgr2=0x7fd0ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:53.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.848+0000 7fd0c5ac1640 1 --2- 172.21.15.44:0/3820720859 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd0ac06c8f0 0x7fd0ac06edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd0c00cf7e0 tx=0x7fd0b4007cf0 comp rx=0 tx=0).stop 2024-04-22T06:31:53.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.848+0000 7fd0c5ac1640 1 -- 172.21.15.44:0/3820720859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0c00c9c50 msgr2=0x7fd0c01520b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:53.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.848+0000 7fd0c5ac1640 1 --2- 172.21.15.44:0/3820720859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0c00c9c50 0x7fd0c01520b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0036040 tx=0x7fd0b0004a40 comp rx=0 tx=0).stop 2024-04-22T06:31:53.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.848+0000 7fd0c5ac1640 1 -- 172.21.15.44:0/3820720859 shutdown_connections 2024-04-22T06:31:53.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.848+0000 7fd0c5ac1640 1 --2- 172.21.15.44:0/3820720859 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd0ac06c8f0 0x7fd0ac06edb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:53.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.848+0000 7fd0c5ac1640 1 --2- 172.21.15.44:0/3820720859 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0c01530e0 0x7fd0c0145090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:53.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.848+0000 7fd0c5ac1640 1 --2- 172.21.15.44:0/3820720859 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0c01598e0 0x7fd0c01481a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:53.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.848+0000 7fd0c5ac1640 1 --2- 172.21.15.44:0/3820720859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0c00c9c50 0x7fd0c01520b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:53.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.848+0000 7fd0c5ac1640 1 -- 172.21.15.44:0/3820720859 >> 172.21.15.44:0/3820720859 conn(0x7fd0c0074cb0 msgr2=0x7fd0c0075090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:53.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.848+0000 7fd0c5ac1640 1 -- 172.21.15.44:0/3820720859 shutdown_connections 2024-04-22T06:31:53.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.848+0000 7fd0c5ac1640 1 -- 172.21.15.44:0/3820720859 wait complete. 2024-04-22T06:31:53.858 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T06:30:54.560256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T06:31:04.612387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T06:31:05.312124+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-22T06:31:09.025550+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-22T06:31:53.858 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T06:31:53.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.964+0000 7ff4eb710640 1 Processor -- start 2024-04-22T06:31:53.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.964+0000 7ff4eb710640 1 -- start start 2024-04-22T06:31:53.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.965+0000 7ff4eb710640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff4e415bc90 0x7ff4e4070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:53.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.965+0000 7ff4eb710640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4e406d4e0 con 0x7ff4e41551e0 2024-04-22T06:31:53.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.965+0000 7ff4eb710640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4e4073380 con 0x7ff4e40711e0 2024-04-22T06:31:53.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.965+0000 7ff4eb710640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7ff4e4070810 con 0x7ff4e415bc90 2024-04-22T06:31:53.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.965+0000 7ff4e94ac640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7ff4e41551e0 0x7ff4e406b930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:38662/0 (socket says 172.21.15.44:38662) 2024-04-22T06:31:53.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.965+0000 7ff4e94ac640 1 -- 172.21.15.44:0/495911358 learned_addr learned my addr 172.21.15.44:0/495911358 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:53.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.965+0000 7ff4e8cab640 1 --2- 172.21.15.44:0/495911358 >> v2:172.21.15.70:3300/0 conn(0x7ff4e415bc90 0x7ff4e4070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:53.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.965+0000 7ff4dbfff640 1 -- 172.21.15.44:0/495911358 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 199310530 0 0) 0x7ff4e406d4e0 con 0x7ff4e41551e0 2024-04-22T06:31:53.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.965+0000 7ff4e8cab640 1 -- 172.21.15.44:0/495911358 >> v1:172.21.15.70:6789/0 conn(0x7ff4e40711e0 legacy=0x7ff4e4165b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:53.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.965+0000 7ff4e8cab640 1 -- 172.21.15.44:0/495911358 >> v1:172.21.15.44:6789/0 conn(0x7ff4e41551e0 legacy=0x7ff4e406b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:53.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.965+0000 7ff4e8cab640 1 -- 172.21.15.44:0/495911358 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e4073660 con 0x7ff4e415bc90 2024-04-22T06:31:53.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.966+0000 7ff4e8cab640 1 --2- 172.21.15.44:0/495911358 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4e415bc90 0x7ff4e4070ca0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff4d40097a0 tx=0x7ff4d402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1a2c6e124d56370 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:53.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.966+0000 7ff4dbfff640 1 -- 172.21.15.44:0/495911358 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d4034070 con 0x7ff4e415bc90 2024-04-22T06:31:53.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.966+0000 7ff4dbfff640 1 -- 172.21.15.44:0/495911358 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4d4030610 con 0x7ff4e415bc90 2024-04-22T06:31:53.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.966+0000 7ff4dbfff640 1 -- 172.21.15.44:0/495911358 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d40309e0 con 0x7ff4e415bc90 2024-04-22T06:31:53.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.966+0000 7ff4eb710640 1 -- 172.21.15.44:0/495911358 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4e415bc90 msgr2=0x7ff4e4070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:53.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.966+0000 7ff4eb710640 1 --2- 172.21.15.44:0/495911358 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4e415bc90 0x7ff4e4070ca0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff4d40097a0 tx=0x7ff4d402b960 comp rx=0 tx=0).stop 2024-04-22T06:31:53.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.966+0000 7ff4eb710640 1 -- 172.21.15.44:0/495911358 shutdown_connections 2024-04-22T06:31:53.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.966+0000 7ff4eb710640 1 --2- 172.21.15.44:0/495911358 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4e415bc90 0x7ff4e4070ca0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:53.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.966+0000 7ff4eb710640 1 -- 172.21.15.44:0/495911358 >> 172.21.15.44:0/495911358 conn(0x7ff4e40ce700 msgr2=0x7ff4e40ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:53.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.967+0000 7ff4eb710640 1 -- 172.21.15.44:0/495911358 shutdown_connections 2024-04-22T06:31:53.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.967+0000 7ff4eb710640 1 -- 172.21.15.44:0/495911358 wait complete. 2024-04-22T06:31:53.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.968+0000 7ff4eb710640 1 Processor -- start 2024-04-22T06:31:53.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.968+0000 7ff4eb710640 1 -- start start 2024-04-22T06:31:53.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.968+0000 7ff4eb710640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4e40711e0 0x7ff4e40cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:53.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.968+0000 7ff4eb710640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4e41551e0 0x7ff4e4159360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:53.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.968+0000 7ff4eb710640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4e415bc90 0x7ff4e41598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:53.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.968+0000 7ff4eb710640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e40cf420 con 0x7ff4e415bc90 2024-04-22T06:31:53.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.968+0000 7ff4eb710640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e4073c20 con 0x7ff4e40711e0 2024-04-22T06:31:53.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.968+0000 7ff4eb710640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ff4e40715f0 con 0x7ff4e41551e0 2024-04-22T06:31:53.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.969+0000 7ff4e9cad640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4e415bc90 0x7ff4e41598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:53.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.969+0000 7ff4e9cad640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4e415bc90 0x7ff4e41598a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55020/0 (socket says 172.21.15.44:55020) 2024-04-22T06:31:53.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.969+0000 7ff4e9cad640 1 -- 172.21.15.44:0/1879702905 learned_addr learned my addr 172.21.15.44:0/1879702905 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:53.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.969+0000 7ff4e8cab640 1 --2- 172.21.15.44:0/1879702905 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4e41551e0 0x7ff4e4159360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:53.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.969+0000 7ff4e94ac640 1 --2- 172.21.15.44:0/1879702905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4e40711e0 0x7ff4e40cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:53.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.969+0000 7ff4e94ac640 1 -- 172.21.15.44:0/1879702905 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4e41551e0 msgr2=0x7ff4e4159360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:53.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.969+0000 7ff4e94ac640 1 --2- 172.21.15.44:0/1879702905 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4e41551e0 0x7ff4e4159360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:53.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.969+0000 7ff4e94ac640 1 -- 172.21.15.44:0/1879702905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4e415bc90 msgr2=0x7ff4e41598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:53.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.969+0000 7ff4e94ac640 1 --2- 172.21.15.44:0/1879702905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4e415bc90 0x7ff4e41598a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:53.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.969+0000 7ff4e94ac640 1 -- 172.21.15.44:0/1879702905 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e415efe0 con 0x7ff4e40711e0 2024-04-22T06:31:53.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.969+0000 7ff4e8cab640 1 --2- 172.21.15.44:0/1879702905 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4e41551e0 0x7ff4e4159360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:31:53.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.969+0000 7ff4e94ac640 1 --2- 172.21.15.44:0/1879702905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4e40711e0 0x7ff4e40cbb90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc00c440 tx=0x7ff4dc00c900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:53.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.969+0000 7ff4d9ffb640 1 -- 172.21.15.44:0/1879702905 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc018070 con 0x7ff4e40711e0 2024-04-22T06:31:53.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.969+0000 7ff4d9ffb640 1 -- 172.21.15.44:0/1879702905 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4dc0074e0 con 0x7ff4e40711e0 2024-04-22T06:31:53.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.969+0000 7ff4eb710640 1 -- 172.21.15.44:0/1879702905 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4e415f2c0 con 0x7ff4e40711e0 2024-04-22T06:31:53.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.969+0000 7ff4d9ffb640 1 -- 172.21.15.44:0/1879702905 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc007690 con 0x7ff4e40711e0 2024-04-22T06:31:53.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.969+0000 7ff4eb710640 1 -- 172.21.15.44:0/1879702905 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4e415d570 con 0x7ff4e40711e0 2024-04-22T06:31:53.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.971+0000 7ff4eb710640 1 -- 172.21.15.44:0/1879702905 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4e4069310 con 0x7ff4e40711e0 2024-04-22T06:31:53.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.971+0000 7ff4d9ffb640 1 -- 172.21.15.44:0/1879702905 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff4dc01b070 con 0x7ff4e40711e0 2024-04-22T06:31:53.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.971+0000 7ff4d9ffb640 1 --2- 172.21.15.44:0/1879702905 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7ff4c806c8f0 0x7ff4c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:53.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.971+0000 7ff4d9ffb640 1 -- 172.21.15.44:0/1879702905 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ff4dc08b0a0 con 0x7ff4e40711e0 2024-04-22T06:31:53.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.971+0000 7ff4e8cab640 1 --2- 172.21.15.44:0/1879702905 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7ff4c806c8f0 0x7ff4c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:53.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.972+0000 7ff4e8cab640 1 --2- 172.21.15.44:0/1879702905 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7ff4c806c8f0 0x7ff4c806edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4036000 tx=0x7ff4d4008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:53.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:53.976+0000 7ff4d9ffb640 1 -- 172.21.15.44:0/1879702905 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff4dc058a80 con 0x7ff4e40711e0 2024-04-22T06:31:54.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.161+0000 7ff4eb710640 1 -- 172.21.15.44:0/1879702905 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff4e406d120 con 0x7ff4e40711e0 2024-04-22T06:31:54.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.161+0000 7ff4d9ffb640 1 -- 172.21.15.44:0/1879702905 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7ff4dc05cd80 con 0x7ff4e40711e0 2024-04-22T06:31:54.160 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-04-22T06:31:54.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.163+0000 7ff4eb710640 1 -- 172.21.15.44:0/1879702905 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7ff4c806c8f0 msgr2=0x7ff4c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:54.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.163+0000 7ff4eb710640 1 --2- 172.21.15.44:0/1879702905 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7ff4c806c8f0 0x7ff4c806edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4036000 tx=0x7ff4d4008030 comp rx=0 tx=0).stop 2024-04-22T06:31:54.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.163+0000 7ff4eb710640 1 -- 172.21.15.44:0/1879702905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4e40711e0 msgr2=0x7ff4e40cbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:54.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.163+0000 7ff4eb710640 1 --2- 172.21.15.44:0/1879702905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4e40711e0 0x7ff4e40cbb90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc00c440 tx=0x7ff4dc00c900 comp rx=0 tx=0).stop 2024-04-22T06:31:54.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.163+0000 7ff4eb710640 1 -- 172.21.15.44:0/1879702905 shutdown_connections 2024-04-22T06:31:54.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.163+0000 7ff4eb710640 1 --2- 172.21.15.44:0/1879702905 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7ff4c806c8f0 0x7ff4c806edb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:54.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.163+0000 7ff4eb710640 1 --2- 172.21.15.44:0/1879702905 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4e41551e0 0x7ff4e4159360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:54.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.163+0000 7ff4eb710640 1 --2- 172.21.15.44:0/1879702905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4e415bc90 0x7ff4e41598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:54.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.163+0000 7ff4eb710640 1 --2- 172.21.15.44:0/1879702905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4e40711e0 0x7ff4e40cbb90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:54.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.164+0000 7ff4eb710640 1 -- 172.21.15.44:0/1879702905 >> 172.21.15.44:0/1879702905 conn(0x7ff4e40ce700 msgr2=0x7ff4e40cd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:54.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.164+0000 7ff4eb710640 1 -- 172.21.15.44:0/1879702905 shutdown_connections 2024-04-22T06:31:54.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.164+0000 7ff4eb710640 1 -- 172.21.15.44:0/1879702905 wait complete. 2024-04-22T06:31:54.173 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T06:31:54.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.278+0000 7f6b6cbbd640 1 Processor -- start 2024-04-22T06:31:54.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.278+0000 7f6b6cbbd640 1 -- start start 2024-04-22T06:31:54.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.279+0000 7f6b6cbbd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6b681551e0 0x7f6b6815bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:54.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.279+0000 7f6b6cbbd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b6806b930 0x7f6b6806bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:54.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.279+0000 7f6b6cbbd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f6b68070ca0 0x7f6b680710f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:54.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.279+0000 7f6b6cbbd640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f6b68070810 con 0x7f6b68070ca0 2024-04-22T06:31:54.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.279+0000 7f6b6cbbd640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f6b6806d4e0 con 0x7f6b681551e0 2024-04-22T06:31:54.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.279+0000 7f6b6cbbd640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f6b68071630 con 0x7f6b6806b930 2024-04-22T06:31:54.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.279+0000 7f6b66d9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f6b68070ca0 0x7f6b680710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:54.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.279+0000 7f6b66d9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f6b68070ca0 0x7f6b680710f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55026/0 (socket says 172.21.15.44:55026) 2024-04-22T06:31:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.279+0000 7f6b65d9b640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b6806b930 0x7f6b6806bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.279+0000 7f6b66d9d640 1 -- 172.21.15.44:0/3915443833 learned_addr learned my addr 172.21.15.44:0/3915443833 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.279+0000 7f6b6659c640 1 --2- 172.21.15.44:0/3915443833 >> v2:172.21.15.70:3300/0 conn(0x7f6b681551e0 0x7f6b6815bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.280+0000 7f6b66d9d640 1 -- 172.21.15.44:0/3915443833 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b6806b930 msgr2=0x7f6b6806bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.280+0000 7f6b66d9d640 1 --2- 172.21.15.44:0/3915443833 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b6806b930 0x7f6b6806bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.280+0000 7f6b66d9d640 1 -- 172.21.15.44:0/3915443833 >> v2:172.21.15.70:3300/0 conn(0x7f6b681551e0 msgr2=0x7f6b6815bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.280+0000 7f6b66d9d640 1 --2- 172.21.15.44:0/3915443833 >> v2:172.21.15.70:3300/0 conn(0x7f6b681551e0 0x7f6b6815bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.280+0000 7f6b66d9d640 1 -- 172.21.15.44:0/3915443833 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b68073380 con 0x7f6b68070ca0 2024-04-22T06:31:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.280+0000 7f6b65d9b640 1 --2- 172.21.15.44:0/3915443833 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b6806b930 0x7f6b6806bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:31:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.280+0000 7f6b6659c640 1 --2- 172.21.15.44:0/3915443833 >> v2:172.21.15.70:3300/0 conn(0x7f6b681551e0 0x7f6b6815bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:31:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.280+0000 7f6b66d9d640 1 --2- 172.21.15.44:0/3915443833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b68070ca0 0x7f6b680710f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6b68068930 tx=0x7f6b5802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9576658acd8a21 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.281+0000 7f6b6559a640 1 -- 172.21.15.44:0/3915443833 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5802ca80 con 0x7f6b68070ca0 2024-04-22T06:31:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.281+0000 7f6b6559a640 1 -- 172.21.15.44:0/3915443833 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b5802cc20 con 0x7f6b68070ca0 2024-04-22T06:31:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.281+0000 7f6b6559a640 1 -- 172.21.15.44:0/3915443833 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b580343f0 con 0x7f6b68070ca0 2024-04-22T06:31:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.281+0000 7f6b6cbbd640 1 -- 172.21.15.44:0/3915443833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b68070ca0 msgr2=0x7f6b680710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:54.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.281+0000 7f6b6cbbd640 1 --2- 172.21.15.44:0/3915443833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b68070ca0 0x7f6b680710f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6b68068930 tx=0x7f6b5802bee0 comp rx=0 tx=0).stop 2024-04-22T06:31:54.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.281+0000 7f6b6cbbd640 1 -- 172.21.15.44:0/3915443833 shutdown_connections 2024-04-22T06:31:54.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.281+0000 7f6b6cbbd640 1 --2- 172.21.15.44:0/3915443833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b68070ca0 0x7f6b680710f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:54.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.281+0000 7f6b6cbbd640 1 --2- 172.21.15.44:0/3915443833 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b6806b930 0x7f6b6806bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:54.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.281+0000 7f6b6cbbd640 1 --2- 172.21.15.44:0/3915443833 >> v2:172.21.15.70:3300/0 conn(0x7f6b681551e0 0x7f6b6815bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:54.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.281+0000 7f6b6cbbd640 1 -- 172.21.15.44:0/3915443833 >> 172.21.15.44:0/3915443833 conn(0x7f6b680ce700 msgr2=0x7f6b680ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:54.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.281+0000 7f6b6cbbd640 1 -- 172.21.15.44:0/3915443833 shutdown_connections 2024-04-22T06:31:54.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.281+0000 7f6b6cbbd640 1 -- 172.21.15.44:0/3915443833 wait complete. 2024-04-22T06:31:54.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.283+0000 7f6b6cbbd640 1 Processor -- start 2024-04-22T06:31:54.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.283+0000 7f6b6cbbd640 1 -- start start 2024-04-22T06:31:54.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.283+0000 7f6b6cbbd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b6806b930 0x7f6b68142cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:54.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.283+0000 7f6b6cbbd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b68070ca0 0x7f6b680cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:54.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.283+0000 7f6b6cbbd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b681551e0 0x7f6b680c5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:54.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.283+0000 7f6b6cbbd640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6b68073c20 con 0x7f6b6806b930 2024-04-22T06:31:54.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.283+0000 7f6b6cbbd640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6b68073940 con 0x7f6b681551e0 2024-04-22T06:31:54.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.283+0000 7f6b6cbbd640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f6b680cf420 con 0x7f6b68070ca0 2024-04-22T06:31:54.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.284+0000 7f6b6659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b6806b930 0x7f6b68142cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:54.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.284+0000 7f6b6659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b6806b930 0x7f6b68142cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55040/0 (socket says 172.21.15.44:55040) 2024-04-22T06:31:54.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.284+0000 7f6b6659c640 1 -- 172.21.15.44:0/4039063950 learned_addr learned my addr 172.21.15.44:0/4039063950 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:54.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.284+0000 7f6b65d9b640 1 --2- 172.21.15.44:0/4039063950 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b68070ca0 0x7f6b680cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:54.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.284+0000 7f6b66d9d640 1 --2- 172.21.15.44:0/4039063950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b681551e0 0x7f6b680c5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:54.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.284+0000 7f6b6659c640 1 -- 172.21.15.44:0/4039063950 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b68070ca0 msgr2=0x7f6b680cdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:54.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.284+0000 7f6b6659c640 1 --2- 172.21.15.44:0/4039063950 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b68070ca0 0x7f6b680cdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:54.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.284+0000 7f6b6659c640 1 -- 172.21.15.44:0/4039063950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b681551e0 msgr2=0x7f6b680c5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:54.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.284+0000 7f6b6659c640 1 --2- 172.21.15.44:0/4039063950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b681551e0 0x7f6b680c5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:54.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.284+0000 7f6b6659c640 1 -- 172.21.15.44:0/4039063950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b680c8750 con 0x7f6b6806b930 2024-04-22T06:31:54.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.284+0000 7f6b6659c640 1 --2- 172.21.15.44:0/4039063950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b6806b930 0x7f6b68142cb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6b6000e990 tx=0x7f6b6000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:54.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.284+0000 7f6b5effd640 1 -- 172.21.15.44:0/4039063950 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b60017070 con 0x7f6b6806b930 2024-04-22T06:31:54.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.284+0000 7f6b5effd640 1 -- 172.21.15.44:0/4039063950 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b6000c9e0 con 0x7f6b6806b930 2024-04-22T06:31:54.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.284+0000 7f6b6cbbd640 1 -- 172.21.15.44:0/4039063950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b681593b0 con 0x7f6b6806b930 2024-04-22T06:31:54.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.284+0000 7f6b5effd640 1 -- 172.21.15.44:0/4039063950 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6000cdb0 con 0x7f6b6806b930 2024-04-22T06:31:54.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.284+0000 7f6b6cbbd640 1 -- 172.21.15.44:0/4039063950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b6815b810 con 0x7f6b6806b930 2024-04-22T06:31:54.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.285+0000 7f6b6cbbd640 1 -- 172.21.15.44:0/4039063950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b680676b0 con 0x7f6b6806b930 2024-04-22T06:31:54.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.286+0000 7f6b5effd640 1 -- 172.21.15.44:0/4039063950 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6b6001a070 con 0x7f6b6806b930 2024-04-22T06:31:54.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.286+0000 7f6b5effd640 1 --2- 172.21.15.44:0/4039063950 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f6b4806c8f0 0x7f6b4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:54.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.287+0000 7f6b5effd640 1 -- 172.21.15.44:0/4039063950 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f6b6008c0e0 con 0x7f6b6806b930 2024-04-22T06:31:54.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.287+0000 7f6b65d9b640 1 --2- 172.21.15.44:0/4039063950 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f6b4806c8f0 0x7f6b4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:54.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.287+0000 7f6b65d9b640 1 --2- 172.21.15.44:0/4039063950 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f6b4806c8f0 0x7f6b4806edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6b50009560 tx=0x7f6b50009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:54.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.291+0000 7f6b5effd640 1 -- 172.21.15.44:0/4039063950 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6b680676b0 con 0x7f6b6806b930 2024-04-22T06:31:54.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.475+0000 7f6b6cbbd640 1 -- 172.21.15.44:0/4039063950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6b6814fdc0 con 0x7f6b6806b930 2024-04-22T06:31:54.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.476+0000 7f6b5effd640 1 -- 172.21.15.44:0/4039063950 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f6b6814fdc0 con 0x7f6b6806b930 2024-04-22T06:31:54.475 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-04-22T06:31:54.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.478+0000 7f6b6cbbd640 1 -- 172.21.15.44:0/4039063950 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f6b4806c8f0 msgr2=0x7f6b4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:54.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.478+0000 7f6b6cbbd640 1 --2- 172.21.15.44:0/4039063950 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f6b4806c8f0 0x7f6b4806edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6b50009560 tx=0x7f6b50009190 comp rx=0 tx=0).stop 2024-04-22T06:31:54.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.478+0000 7f6b6cbbd640 1 -- 172.21.15.44:0/4039063950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b6806b930 msgr2=0x7f6b68142cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:54.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.478+0000 7f6b6cbbd640 1 --2- 172.21.15.44:0/4039063950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b6806b930 0x7f6b68142cb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6b6000e990 tx=0x7f6b6000ee50 comp rx=0 tx=0).stop 2024-04-22T06:31:54.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.478+0000 7f6b6cbbd640 1 -- 172.21.15.44:0/4039063950 shutdown_connections 2024-04-22T06:31:54.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.478+0000 7f6b6cbbd640 1 --2- 172.21.15.44:0/4039063950 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f6b4806c8f0 0x7f6b4806edb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:54.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.478+0000 7f6b6cbbd640 1 --2- 172.21.15.44:0/4039063950 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b68070ca0 0x7f6b680cdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:54.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.478+0000 7f6b6cbbd640 1 --2- 172.21.15.44:0/4039063950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b681551e0 0x7f6b680c5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:54.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.478+0000 7f6b6cbbd640 1 --2- 172.21.15.44:0/4039063950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b6806b930 0x7f6b68142cb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:54.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.478+0000 7f6b6cbbd640 1 -- 172.21.15.44:0/4039063950 >> 172.21.15.44:0/4039063950 conn(0x7f6b680ce700 msgr2=0x7f6b680cba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:54.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.478+0000 7f6b6cbbd640 1 -- 172.21.15.44:0/4039063950 shutdown_connections 2024-04-22T06:31:54.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.478+0000 7f6b6cbbd640 1 -- 172.21.15.44:0/4039063950 wait complete. 2024-04-22T06:31:54.488 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T06:31:54.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.592+0000 7f8430211640 1 Processor -- start 2024-04-22T06:31:54.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.593+0000 7f8430211640 1 -- start start 2024-04-22T06:31:54.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.593+0000 7f8430211640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f84280c4ce0 0x7f84281599a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:54.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.593+0000 7f8430211640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f84280ce150 0x7f84280c50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:54.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.593+0000 7f8430211640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84280c5600 con 0x7f842806fbf0 2024-04-22T06:31:54.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.593+0000 7f8430211640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f842815c690 con 0x7f84280ce150 2024-04-22T06:31:54.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.593+0000 7f8430211640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f842815c930 con 0x7f84280c4ce0 2024-04-22T06:31:54.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.593+0000 7f842d7ac640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f84280ce150 0x7f84280c50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:54.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.593+0000 7f842e7ae640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f842806fbf0 0x7f8428070060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:38668/0 (socket says 172.21.15.44:38668) 2024-04-22T06:31:54.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.593+0000 7f842e7ae640 1 -- 172.21.15.44:0/702248378 learned_addr learned my addr 172.21.15.44:0/702248378 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:54.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.594+0000 7f842dfad640 1 --2- 172.21.15.44:0/702248378 >> v2:172.21.15.70:3300/0 conn(0x7f84280c4ce0 0x7f84281599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:54.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.594+0000 7f842d7ac640 1 -- 172.21.15.44:0/702248378 >> v2:172.21.15.70:3300/0 conn(0x7f84280c4ce0 msgr2=0x7f84281599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:54.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.594+0000 7f842d7ac640 1 --2- 172.21.15.44:0/702248378 >> v2:172.21.15.70:3300/0 conn(0x7f84280c4ce0 0x7f84281599a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:54.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.594+0000 7f842d7ac640 1 -- 172.21.15.44:0/702248378 >> v1:172.21.15.44:6789/0 conn(0x7f842806fbf0 legacy=0x7f8428070060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:54.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.594+0000 7f842cfab640 1 -- 172.21.15.44:0/702248378 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4059348750 0 0) 0x7f8418001490 con 0x7f842806fbf0 2024-04-22T06:31:54.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.594+0000 7f842d7ac640 1 -- 172.21.15.44:0/702248378 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f842815cbf0 con 0x7f84280ce150 2024-04-22T06:31:54.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.594+0000 7f842dfad640 1 --2- 172.21.15.44:0/702248378 >> v2:172.21.15.70:3300/0 conn(0x7f84280c4ce0 0x7f84281599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:31:54.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.595+0000 7f842d7ac640 1 --2- 172.21.15.44:0/702248378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84280ce150 0x7f84280c50c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f842806cde0 tx=0x7f841c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8103fd3255ec9b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:54.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.595+0000 7f842cfab640 1 -- 172.21.15.44:0/702248378 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c035020 con 0x7f84280ce150 2024-04-22T06:31:54.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.595+0000 7f842cfab640 1 -- 172.21.15.44:0/702248378 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f841c031830 con 0x7f84280ce150 2024-04-22T06:31:54.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.595+0000 7f842cfab640 1 -- 172.21.15.44:0/702248378 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c031c00 con 0x7f84280ce150 2024-04-22T06:31:54.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.595+0000 7f8430211640 1 -- 172.21.15.44:0/702248378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84280ce150 msgr2=0x7f84280c50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:54.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.595+0000 7f8430211640 1 --2- 172.21.15.44:0/702248378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84280ce150 0x7f84280c50c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f842806cde0 tx=0x7f841c02b8d0 comp rx=0 tx=0).stop 2024-04-22T06:31:54.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.595+0000 7f8430211640 1 -- 172.21.15.44:0/702248378 shutdown_connections 2024-04-22T06:31:54.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.595+0000 7f8430211640 1 --2- 172.21.15.44:0/702248378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84280ce150 0x7f84280c50c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:54.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.595+0000 7f8430211640 1 --2- 172.21.15.44:0/702248378 >> v2:172.21.15.70:3300/0 conn(0x7f84280c4ce0 0x7f84281599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:54.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.595+0000 7f8430211640 1 -- 172.21.15.44:0/702248378 >> 172.21.15.44:0/702248378 conn(0x7f84280cd630 msgr2=0x7f84280cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:54.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.595+0000 7f8430211640 1 -- 172.21.15.44:0/702248378 shutdown_connections 2024-04-22T06:31:54.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.595+0000 7f8430211640 1 -- 172.21.15.44:0/702248378 wait complete. 2024-04-22T06:31:54.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.596+0000 7f8430211640 1 Processor -- start 2024-04-22T06:31:54.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.597+0000 7f8430211640 1 -- start start 2024-04-22T06:31:54.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.597+0000 7f8430211640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f842806fbf0 0x7f842815b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:54.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.597+0000 7f8430211640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84280c4ce0 0x7f84281465a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:54.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.597+0000 7f8430211640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f84280ce150 0x7f84280cc590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:54.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.597+0000 7f8430211640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f84281524d0 con 0x7f842806fbf0 2024-04-22T06:31:54.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.597+0000 7f8430211640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f842815d1b0 con 0x7f84280c4ce0 2024-04-22T06:31:54.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.597+0000 7f8430211640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8428152650 con 0x7f84280ce150 2024-04-22T06:31:54.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.597+0000 7f842dfad640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f842806fbf0 0x7f842815b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:54.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.597+0000 7f842dfad640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f842806fbf0 0x7f842815b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55052/0 (socket says 172.21.15.44:55052) 2024-04-22T06:31:54.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.597+0000 7f842dfad640 1 -- 172.21.15.44:0/2880108383 learned_addr learned my addr 172.21.15.44:0/2880108383 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:54.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.597+0000 7f842d7ac640 1 --2- 172.21.15.44:0/2880108383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84280c4ce0 0x7f84281465a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:54.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.597+0000 7f842e7ae640 1 --2- 172.21.15.44:0/2880108383 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f84280ce150 0x7f84280cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:54.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.598+0000 7f842dfad640 1 -- 172.21.15.44:0/2880108383 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f84280ce150 msgr2=0x7f84280cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:54.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.598+0000 7f842dfad640 1 --2- 172.21.15.44:0/2880108383 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f84280ce150 0x7f84280cc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:54.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.598+0000 7f842dfad640 1 -- 172.21.15.44:0/2880108383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84280c4ce0 msgr2=0x7f84281465a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:54.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.598+0000 7f842dfad640 1 --2- 172.21.15.44:0/2880108383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84280c4ce0 0x7f84281465a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:54.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.598+0000 7f842dfad640 1 -- 172.21.15.44:0/2880108383 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f842815a150 con 0x7f842806fbf0 2024-04-22T06:31:54.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.598+0000 7f842dfad640 1 --2- 172.21.15.44:0/2880108383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f842806fbf0 0x7f842815b680 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f842000ebe0 tx=0x7f842000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:54.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.598+0000 7f84267fc640 1 -- 172.21.15.44:0/2880108383 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8420017070 con 0x7f842806fbf0 2024-04-22T06:31:54.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.598+0000 7f84267fc640 1 -- 172.21.15.44:0/2880108383 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f842000cc30 con 0x7f842806fbf0 2024-04-22T06:31:54.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.598+0000 7f8430211640 1 -- 172.21.15.44:0/2880108383 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84281f0af0 con 0x7f842806fbf0 2024-04-22T06:31:54.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.598+0000 7f84267fc640 1 -- 172.21.15.44:0/2880108383 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8420013470 con 0x7f842806fbf0 2024-04-22T06:31:54.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.598+0000 7f8430211640 1 -- 172.21.15.44:0/2880108383 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84281f44e0 con 0x7f842806fbf0 2024-04-22T06:31:54.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.599+0000 7f8430211640 1 -- 172.21.15.44:0/2880108383 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8428069310 con 0x7f842806fbf0 2024-04-22T06:31:54.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.600+0000 7f84267fc640 1 -- 172.21.15.44:0/2880108383 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f842001a070 con 0x7f842806fbf0 2024-04-22T06:31:54.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.600+0000 7f84267fc640 1 --2- 172.21.15.44:0/2880108383 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f841406c8f0 0x7f841406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:54.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.600+0000 7f84267fc640 1 -- 172.21.15.44:0/2880108383 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f842008c190 con 0x7f842806fbf0 2024-04-22T06:31:54.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.600+0000 7f842d7ac640 1 --2- 172.21.15.44:0/2880108383 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f841406c8f0 0x7f841406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:54.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.601+0000 7f842d7ac640 1 --2- 172.21.15.44:0/2880108383 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f841406c8f0 0x7f841406edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f841c036000 tx=0x7f841c03c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:54.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.604+0000 7f84267fc640 1 -- 172.21.15.44:0/2880108383 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8420059b70 con 0x7f842806fbf0 2024-04-22T06:31:54.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.790+0000 7f8430211640 1 -- 172.21.15.44:0/2880108383 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8428150200 con 0x7f842806fbf0 2024-04-22T06:31:54.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.791+0000 7f84267fc640 1 -- 172.21.15.44:0/2880108383 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f842005de70 con 0x7f842806fbf0 2024-04-22T06:31:54.790 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-04-22T06:31:54.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.793+0000 7f8430211640 1 -- 172.21.15.44:0/2880108383 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f841406c8f0 msgr2=0x7f841406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:54.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.793+0000 7f8430211640 1 --2- 172.21.15.44:0/2880108383 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f841406c8f0 0x7f841406edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f841c036000 tx=0x7f841c03c040 comp rx=0 tx=0).stop 2024-04-22T06:31:54.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.793+0000 7f8430211640 1 -- 172.21.15.44:0/2880108383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f842806fbf0 msgr2=0x7f842815b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:54.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.793+0000 7f8430211640 1 --2- 172.21.15.44:0/2880108383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f842806fbf0 0x7f842815b680 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f842000ebe0 tx=0x7f842000c520 comp rx=0 tx=0).stop 2024-04-22T06:31:54.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.793+0000 7f8430211640 1 -- 172.21.15.44:0/2880108383 shutdown_connections 2024-04-22T06:31:54.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.793+0000 7f8430211640 1 --2- 172.21.15.44:0/2880108383 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f841406c8f0 0x7f841406edb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:54.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.793+0000 7f8430211640 1 --2- 172.21.15.44:0/2880108383 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f84280ce150 0x7f84280cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:54.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.793+0000 7f8430211640 1 --2- 172.21.15.44:0/2880108383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84280c4ce0 0x7f84281465a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:54.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.793+0000 7f8430211640 1 --2- 172.21.15.44:0/2880108383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f842806fbf0 0x7f842815b680 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:54.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.793+0000 7f8430211640 1 -- 172.21.15.44:0/2880108383 >> 172.21.15.44:0/2880108383 conn(0x7f84280cd630 msgr2=0x7f8428070060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:54.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.793+0000 7f8430211640 1 -- 172.21.15.44:0/2880108383 shutdown_connections 2024-04-22T06:31:54.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.793+0000 7f8430211640 1 -- 172.21.15.44:0/2880108383 wait complete. 2024-04-22T06:31:54.802 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-22T06:31:54.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.907+0000 7ffa6ba81640 1 Processor -- start 2024-04-22T06:31:54.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.907+0000 7ffa6ba81640 1 -- start start 2024-04-22T06:31:54.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.908+0000 7ffa6ba81640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffa64155570 0x7ffa640cb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:54.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.908+0000 7ffa6ba81640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ffa640cc8e0 0x7ffa64165ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:54.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.908+0000 7ffa6ba81640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa640ccd50 con 0x7ffa640cc4a0 2024-04-22T06:31:54.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.908+0000 7ffa6ba81640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7ffa6406c8a0 con 0x7ffa640cc8e0 2024-04-22T06:31:54.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.908+0000 7ffa6ba81640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ffa6415c1d0 con 0x7ffa64155570 2024-04-22T06:31:54.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.908+0000 7ffa6a01e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ffa640cc8e0 0x7ffa64165ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:54.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.908+0000 7ffa6981d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffa64155570 0x7ffa640cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:54.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.908+0000 7ffa6a01e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ffa640cc8e0 0x7ffa64165ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55062/0 (socket says 172.21.15.44:55062) 2024-04-22T06:31:54.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.908+0000 7ffa6981d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffa64155570 0x7ffa640cb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.44:45512/0 (socket says 172.21.15.44:45512) 2024-04-22T06:31:54.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.908+0000 7ffa6a01e640 1 -- 172.21.15.44:0/4104160987 learned_addr learned my addr 172.21.15.44:0/4104160987 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:54.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.908+0000 7ffa6901c640 1 --1- 172.21.15.44:0/4104160987 >> v1:172.21.15.70:6789/0 conn(0x7ffa640cc4a0 0x7ffa6406cd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.44:36458/0 (socket says 172.21.15.44:36458) 2024-04-22T06:31:54.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.909+0000 7ffa6981d640 1 -- 172.21.15.44:0/4104160987 >> v2:172.21.15.44:3300/0 conn(0x7ffa640cc8e0 msgr2=0x7ffa64165ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:54.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.909+0000 7ffa6981d640 1 --2- 172.21.15.44:0/4104160987 >> v2:172.21.15.44:3300/0 conn(0x7ffa640cc8e0 0x7ffa64165ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:54.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.909+0000 7ffa6881b640 1 -- 172.21.15.44:0/4104160987 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4001820569 0 0) 0x7ffa58001110 con 0x7ffa640cc4a0 2024-04-22T06:31:54.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.909+0000 7ffa6981d640 1 -- 172.21.15.44:0/4104160987 >> v1:172.21.15.70:6789/0 conn(0x7ffa640cc4a0 legacy=0x7ffa6406cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:54.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.909+0000 7ffa6981d640 1 -- 172.21.15.44:0/4104160987 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa6415b650 con 0x7ffa64155570 2024-04-22T06:31:54.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.909+0000 7ffa6981d640 1 --2- 172.21.15.44:0/4104160987 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffa64155570 0x7ffa640cb990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c0097a0 tx=0x7ffa5c02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcd4d1d0bce60c6c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:54.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.909+0000 7ffa6881b640 1 -- 172.21.15.44:0/4104160987 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c0315a0 con 0x7ffa64155570 2024-04-22T06:31:54.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.910+0000 7ffa6881b640 1 -- 172.21.15.44:0/4104160987 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa5c037070 con 0x7ffa64155570 2024-04-22T06:31:54.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.910+0000 7ffa6881b640 1 -- 172.21.15.44:0/4104160987 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c031960 con 0x7ffa64155570 2024-04-22T06:31:54.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.910+0000 7ffa6ba81640 1 -- 172.21.15.44:0/4104160987 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffa64155570 msgr2=0x7ffa640cb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:54.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.910+0000 7ffa6ba81640 1 --2- 172.21.15.44:0/4104160987 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffa64155570 0x7ffa640cb990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c0097a0 tx=0x7ffa5c02b840 comp rx=0 tx=0).stop 2024-04-22T06:31:54.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.910+0000 7ffa6ba81640 1 -- 172.21.15.44:0/4104160987 shutdown_connections 2024-04-22T06:31:54.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.910+0000 7ffa6ba81640 1 --2- 172.21.15.44:0/4104160987 >> v2:172.21.15.44:3300/0 conn(0x7ffa640cc8e0 0x7ffa64165ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:54.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.910+0000 7ffa6ba81640 1 --2- 172.21.15.44:0/4104160987 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffa64155570 0x7ffa640cb990 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:54.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.910+0000 7ffa6ba81640 1 -- 172.21.15.44:0/4104160987 >> 172.21.15.44:0/4104160987 conn(0x7ffa64075870 msgr2=0x7ffa64073020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:54.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.910+0000 7ffa6ba81640 1 -- 172.21.15.44:0/4104160987 shutdown_connections 2024-04-22T06:31:54.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.910+0000 7ffa6ba81640 1 -- 172.21.15.44:0/4104160987 wait complete. 2024-04-22T06:31:54.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.911+0000 7ffa6ba81640 1 Processor -- start 2024-04-22T06:31:54.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.911+0000 7ffa6ba81640 1 -- start start 2024-04-22T06:31:54.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.912+0000 7ffa6ba81640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa640cc4a0 0x7ffa641583e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:54.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.912+0000 7ffa6ba81640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffa640cc8e0 0x7ffa641f43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:54.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.912+0000 7ffa6ba81640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa64155570 0x7ffa640c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:54.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.912+0000 7ffa6ba81640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6415bef0 con 0x7ffa64155570 2024-04-22T06:31:54.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.912+0000 7ffa6ba81640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6415bc10 con 0x7ffa640cc4a0 2024-04-22T06:31:54.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.912+0000 7ffa6ba81640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ffa64155950 con 0x7ffa640cc8e0 2024-04-22T06:31:54.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.912+0000 7ffa6a01e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa64155570 0x7ffa640c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:54.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.912+0000 7ffa6a01e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa64155570 0x7ffa640c7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55072/0 (socket says 172.21.15.44:55072) 2024-04-22T06:31:54.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.912+0000 7ffa6a01e640 1 -- 172.21.15.44:0/1932733766 learned_addr learned my addr 172.21.15.44:0/1932733766 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:54.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.912+0000 7ffa6981d640 1 --2- 172.21.15.44:0/1932733766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa640cc4a0 0x7ffa641583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:54.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.912+0000 7ffa6901c640 1 --2- 172.21.15.44:0/1932733766 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffa640cc8e0 0x7ffa641f43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:54.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.912+0000 7ffa6a01e640 1 -- 172.21.15.44:0/1932733766 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffa640cc8e0 msgr2=0x7ffa641f43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:54.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.912+0000 7ffa6a01e640 1 --2- 172.21.15.44:0/1932733766 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffa640cc8e0 0x7ffa641f43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:54.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.912+0000 7ffa6a01e640 1 -- 172.21.15.44:0/1932733766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa640cc4a0 msgr2=0x7ffa641583e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:54.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.912+0000 7ffa6a01e640 1 --2- 172.21.15.44:0/1932733766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa640cc4a0 0x7ffa641583e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:54.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.912+0000 7ffa6a01e640 1 -- 172.21.15.44:0/1932733766 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa640c8ae0 con 0x7ffa64155570 2024-04-22T06:31:54.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.912+0000 7ffa6981d640 1 --2- 172.21.15.44:0/1932733766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa640cc4a0 0x7ffa641583e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:31:54.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.913+0000 7ffa6a01e640 1 --2- 172.21.15.44:0/1932733766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa64155570 0x7ffa640c7ea0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ffa5400efc0 tx=0x7ffa5400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:54.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.913+0000 7ffa627fc640 1 -- 172.21.15.44:0/1932733766 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5400ccc0 con 0x7ffa64155570 2024-04-22T06:31:54.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.913+0000 7ffa627fc640 1 -- 172.21.15.44:0/1932733766 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa54003990 con 0x7ffa64155570 2024-04-22T06:31:54.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.913+0000 7ffa6ba81640 1 -- 172.21.15.44:0/1932733766 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa641574f0 con 0x7ffa64155570 2024-04-22T06:31:54.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.913+0000 7ffa627fc640 1 -- 172.21.15.44:0/1932733766 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5400ce60 con 0x7ffa64155570 2024-04-22T06:31:54.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.913+0000 7ffa6ba81640 1 -- 172.21.15.44:0/1932733766 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa6415e870 con 0x7ffa64155570 2024-04-22T06:31:54.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.914+0000 7ffa6ba81640 1 -- 172.21.15.44:0/1932733766 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa64068cc0 con 0x7ffa64155570 2024-04-22T06:31:54.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.914+0000 7ffa627fc640 1 -- 172.21.15.44:0/1932733766 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffa54005d10 con 0x7ffa64155570 2024-04-22T06:31:54.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.915+0000 7ffa627fc640 1 --2- 172.21.15.44:0/1932733766 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7ffa5006c920 0x7ffa5006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:54.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.915+0000 7ffa627fc640 1 -- 172.21.15.44:0/1932733766 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ffa5408c2e0 con 0x7ffa64155570 2024-04-22T06:31:54.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.915+0000 7ffa6981d640 1 --2- 172.21.15.44:0/1932733766 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7ffa5006c920 0x7ffa5006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:54.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.915+0000 7ffa6981d640 1 --2- 172.21.15.44:0/1932733766 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7ffa5006c920 0x7ffa5006ede0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c036000 tx=0x7ffa5c03d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:54.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:54.919+0000 7ffa627fc640 1 -- 172.21.15.44:0/1932733766 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffa54022020 con 0x7ffa64155570 2024-04-22T06:31:55.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.102+0000 7ffa6ba81640 1 -- 172.21.15.44:0/1932733766 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ffa640b63d0 con 0x7ffa64155570 2024-04-22T06:31:55.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.102+0000 7ffa627fc640 1 -- 172.21.15.44:0/1932733766 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7ffa54010040 con 0x7ffa64155570 2024-04-22T06:31:55.101 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-04-22T06:31:55.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.104+0000 7ffa6ba81640 1 -- 172.21.15.44:0/1932733766 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7ffa5006c920 msgr2=0x7ffa5006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:55.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.104+0000 7ffa6ba81640 1 --2- 172.21.15.44:0/1932733766 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7ffa5006c920 0x7ffa5006ede0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c036000 tx=0x7ffa5c03d040 comp rx=0 tx=0).stop 2024-04-22T06:31:55.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.104+0000 7ffa6ba81640 1 -- 172.21.15.44:0/1932733766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa64155570 msgr2=0x7ffa640c7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:55.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.104+0000 7ffa6ba81640 1 --2- 172.21.15.44:0/1932733766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa64155570 0x7ffa640c7ea0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ffa5400efc0 tx=0x7ffa5400c490 comp rx=0 tx=0).stop 2024-04-22T06:31:55.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.104+0000 7ffa6ba81640 1 -- 172.21.15.44:0/1932733766 shutdown_connections 2024-04-22T06:31:55.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.104+0000 7ffa6ba81640 1 --2- 172.21.15.44:0/1932733766 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7ffa5006c920 0x7ffa5006ede0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:55.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.104+0000 7ffa6ba81640 1 --2- 172.21.15.44:0/1932733766 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffa640cc8e0 0x7ffa641f43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:55.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.104+0000 7ffa6ba81640 1 --2- 172.21.15.44:0/1932733766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa64155570 0x7ffa640c7ea0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:55.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.104+0000 7ffa6ba81640 1 --2- 172.21.15.44:0/1932733766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa640cc4a0 0x7ffa641583e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:55.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.104+0000 7ffa6ba81640 1 -- 172.21.15.44:0/1932733766 >> 172.21.15.44:0/1932733766 conn(0x7ffa64075870 msgr2=0x7ffa641539e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:55.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.104+0000 7ffa6ba81640 1 -- 172.21.15.44:0/1932733766 shutdown_connections 2024-04-22T06:31:55.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.104+0000 7ffa6ba81640 1 -- 172.21.15.44:0/1932733766 wait complete. 2024-04-22T06:31:55.114 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T06:31:55.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.221+0000 7f065de5c640 1 Processor -- start 2024-04-22T06:31:55.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.221+0000 7f065de5c640 1 -- start start 2024-04-22T06:31:55.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.221+0000 7f065de5c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0658155340 0x7f06580cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:55.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.221+0000 7f065de5c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f06580cc310 0x7f06580cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:55.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.221+0000 7f065de5c640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06580c5b70 con 0x7f065815e610 2024-04-22T06:31:55.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.221+0000 7f065de5c640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f065815b020 con 0x7f06580cc310 2024-04-22T06:31:55.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.221+0000 7f065de5c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f0658155720 con 0x7f0658155340 2024-04-22T06:31:55.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.222+0000 7f0657fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f06580cc310 0x7f06580cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:55.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.222+0000 7f0656ffd640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f065815e610 0x7f065815b800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:38672/0 (socket says 172.21.15.44:38672) 2024-04-22T06:31:55.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.222+0000 7f0657fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f06580cc310 0x7f06580cb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55100/0 (socket says 172.21.15.44:55100) 2024-04-22T06:31:55.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.222+0000 7f0656ffd640 1 -- 172.21.15.44:0/2005927977 learned_addr learned my addr 172.21.15.44:0/2005927977 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:55.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.222+0000 7f06577fe640 1 --2- 172.21.15.44:0/2005927977 >> v2:172.21.15.70:3300/0 conn(0x7f0658155340 0x7f06580cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:55.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.222+0000 7f0657fff640 1 -- 172.21.15.44:0/2005927977 >> v2:172.21.15.70:3300/0 conn(0x7f0658155340 msgr2=0x7f06580cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:55.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.222+0000 7f0657fff640 1 --2- 172.21.15.44:0/2005927977 >> v2:172.21.15.70:3300/0 conn(0x7f0658155340 0x7f06580cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:55.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.222+0000 7f0657fff640 1 -- 172.21.15.44:0/2005927977 >> v1:172.21.15.44:6789/0 conn(0x7f065815e610 legacy=0x7f065815b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:55.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.222+0000 7f0657fff640 1 -- 172.21.15.44:0/2005927977 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f065815eb50 con 0x7f06580cc310 2024-04-22T06:31:55.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.222+0000 7f06567fc640 1 -- 172.21.15.44:0/2005927977 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 547503287 0 0) 0x7f06580c5b70 con 0x7f065815e610 2024-04-22T06:31:55.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.223+0000 7f0657fff640 1 --2- 172.21.15.44:0/2005927977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06580cc310 0x7f06580cb500 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f06580ce190 tx=0x7f065002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c251efb5995c82c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:55.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.223+0000 7f06567fc640 1 -- 172.21.15.44:0/2005927977 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0650039070 con 0x7f06580cc310 2024-04-22T06:31:55.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.223+0000 7f06567fc640 1 -- 172.21.15.44:0/2005927977 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f065002cc80 con 0x7f06580cc310 2024-04-22T06:31:55.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.223+0000 7f06567fc640 1 -- 172.21.15.44:0/2005927977 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0650034660 con 0x7f06580cc310 2024-04-22T06:31:55.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.223+0000 7f065de5c640 1 -- 172.21.15.44:0/2005927977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06580cc310 msgr2=0x7f06580cb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:55.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.223+0000 7f065de5c640 1 --2- 172.21.15.44:0/2005927977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06580cc310 0x7f06580cb500 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f06580ce190 tx=0x7f065002bed0 comp rx=0 tx=0).stop 2024-04-22T06:31:55.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.223+0000 7f065de5c640 1 -- 172.21.15.44:0/2005927977 shutdown_connections 2024-04-22T06:31:55.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.223+0000 7f065de5c640 1 --2- 172.21.15.44:0/2005927977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06580cc310 0x7f06580cb500 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:55.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.223+0000 7f065de5c640 1 --2- 172.21.15.44:0/2005927977 >> v2:172.21.15.70:3300/0 conn(0x7f0658155340 0x7f06580cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:55.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.223+0000 7f065de5c640 1 -- 172.21.15.44:0/2005927977 >> 172.21.15.44:0/2005927977 conn(0x7f065806c560 msgr2=0x7f0658074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:55.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.223+0000 7f065de5c640 1 -- 172.21.15.44:0/2005927977 shutdown_connections 2024-04-22T06:31:55.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.223+0000 7f065de5c640 1 -- 172.21.15.44:0/2005927977 wait complete. 2024-04-22T06:31:55.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.225+0000 7f065de5c640 1 Processor -- start 2024-04-22T06:31:55.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.225+0000 7f065de5c640 1 -- start start 2024-04-22T06:31:55.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.225+0000 7f065de5c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f06580cc310 0x7f065814f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:55.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.226+0000 7f065de5c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0658155340 0x7f06581f0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:55.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.226+0000 7f065de5c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f065815e610 0x7f065814a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:55.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.226+0000 7f065de5c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f065815f3f0 con 0x7f065815e610 2024-04-22T06:31:55.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.226+0000 7f065de5c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f065815f110 con 0x7f0658155340 2024-04-22T06:31:55.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.226+0000 7f065de5c640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f065815b2e0 con 0x7f06580cc310 2024-04-22T06:31:55.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.226+0000 7f0657fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f065815e610 0x7f065814a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:55.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.226+0000 7f0657fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f065815e610 0x7f065814a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55102/0 (socket says 172.21.15.44:55102) 2024-04-22T06:31:55.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.226+0000 7f0657fff640 1 -- 172.21.15.44:0/3972641946 learned_addr learned my addr 172.21.15.44:0/3972641946 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:55.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.226+0000 7f0656ffd640 1 --2- 172.21.15.44:0/3972641946 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0658155340 0x7f06581f0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:55.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.226+0000 7f06577fe640 1 --2- 172.21.15.44:0/3972641946 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f06580cc310 0x7f065814f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:55.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.226+0000 7f0657fff640 1 -- 172.21.15.44:0/3972641946 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f06580cc310 msgr2=0x7f065814f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:55.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.226+0000 7f0657fff640 1 --2- 172.21.15.44:0/3972641946 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f06580cc310 0x7f065814f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:55.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.226+0000 7f0657fff640 1 -- 172.21.15.44:0/3972641946 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0658155340 msgr2=0x7f06581f0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:55.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.226+0000 7f0657fff640 1 --2- 172.21.15.44:0/3972641946 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0658155340 0x7f06581f0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:55.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.226+0000 7f0657fff640 1 -- 172.21.15.44:0/3972641946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f065815e3c0 con 0x7f065815e610 2024-04-22T06:31:55.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.226+0000 7f0656ffd640 1 --2- 172.21.15.44:0/3972641946 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0658155340 0x7f06581f0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:31:55.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.226+0000 7f0657fff640 1 --2- 172.21.15.44:0/3972641946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f065815e610 0x7f065814a850 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f065002c4e0 tx=0x7f0650004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:55.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.226+0000 7f063ffff640 1 -- 172.21.15.44:0/3972641946 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0650039050 con 0x7f065815e610 2024-04-22T06:31:55.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.226+0000 7f063ffff640 1 -- 172.21.15.44:0/3972641946 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f065003a040 con 0x7f065815e610 2024-04-22T06:31:55.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.226+0000 7f063ffff640 1 -- 172.21.15.44:0/3972641946 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0650009450 con 0x7f065815e610 2024-04-22T06:31:55.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.226+0000 7f065de5c640 1 -- 172.21.15.44:0/3972641946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0658150020 con 0x7f065815e610 2024-04-22T06:31:55.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.227+0000 7f065de5c640 1 -- 172.21.15.44:0/3972641946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06581f4370 con 0x7f065815e610 2024-04-22T06:31:55.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.227+0000 7f065de5c640 1 -- 172.21.15.44:0/3972641946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0658068cc0 con 0x7f065815e610 2024-04-22T06:31:55.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.228+0000 7f063ffff640 1 -- 172.21.15.44:0/3972641946 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0650047020 con 0x7f065815e610 2024-04-22T06:31:55.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.228+0000 7f063ffff640 1 --2- 172.21.15.44:0/3972641946 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f064406c8f0 0x7f064406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:55.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.228+0000 7f063ffff640 1 -- 172.21.15.44:0/3972641946 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f06500ae190 con 0x7f065815e610 2024-04-22T06:31:55.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.229+0000 7f06577fe640 1 --2- 172.21.15.44:0/3972641946 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f064406c8f0 0x7f064406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:55.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.229+0000 7f06577fe640 1 --2- 172.21.15.44:0/3972641946 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f064406c8f0 0x7f064406edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0648009b40 tx=0x7f0648009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:55.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.233+0000 7f063ffff640 1 -- 172.21.15.44:0/3972641946 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f065004e020 con 0x7f065815e610 2024-04-22T06:31:55.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.457+0000 7f065de5c640 1 -- 172.21.15.44:0/3972641946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0658154b60 con 0x7f065815e610 2024-04-22T06:31:55.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.457+0000 7f063ffff640 1 -- 172.21.15.44:0/3972641946 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6901 (secure 0 0 0) 0x7f065007bad0 con 0x7f065815e610 2024-04-22T06:31:55.456 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:31:55.457 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/1779271860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":1779271860},{"type":"v1","addr":"172.21.15.70:6836","nonce":1779271860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T06:31:06.356312+0000","modified":"2024-04-22T06:31:50.151090+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4308,"mds_1":4305,"mds_2":4293,"mds_3":4290,"mds_4":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":4,"incarnation":17,"state":"up:creating","state_seq":1,"addr":"172.21.15.44:6834/1497302315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1497302315},{"type":"v1","addr":"172.21.15.44:6834","nonce":1497302315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4290":{"gid":4290,"name":"a","rank":3,"incarnation":15,"state":"up:active","state_seq":14,"addr":"172.21.15.44:6835/2383621014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2383621014},{"type":"v1","addr":"172.21.15.44:6835","nonce":2383621014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"e","rank":2,"incarnation":13,"state":"up:active","state_seq":12,"addr":"172.21.15.44:6837/915960587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":915960587},{"type":"v1","addr":"172.21.15.44:6837","nonce":915960587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4305":{"gid":4305,"name":"b","rank":1,"incarnation":9,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6838/924955785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":924955785},{"type":"v1","addr":"172.21.15.70:6838","nonce":924955785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4308":{"gid":4308,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.70:6839/2915150040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6837","nonce":2915150040},{"type":"v1","addr":"172.21.15.70:6839","nonce":2915150040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4308,"qdb_cluster":[4308,4305,4293,4290]},"id":1}]} 2024-04-22T06:31:55.457 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 17 2024-04-22T06:31:55.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.459+0000 7f065de5c640 1 -- 172.21.15.44:0/3972641946 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f064406c8f0 msgr2=0x7f064406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:55.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.459+0000 7f065de5c640 1 --2- 172.21.15.44:0/3972641946 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f064406c8f0 0x7f064406edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0648009b40 tx=0x7f0648009190 comp rx=0 tx=0).stop 2024-04-22T06:31:55.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.459+0000 7f065de5c640 1 -- 172.21.15.44:0/3972641946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f065815e610 msgr2=0x7f065814a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:55.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.459+0000 7f065de5c640 1 --2- 172.21.15.44:0/3972641946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f065815e610 0x7f065814a850 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f065002c4e0 tx=0x7f0650004a40 comp rx=0 tx=0).stop 2024-04-22T06:31:55.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.460+0000 7f065de5c640 1 -- 172.21.15.44:0/3972641946 shutdown_connections 2024-04-22T06:31:55.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.460+0000 7f065de5c640 1 --2- 172.21.15.44:0/3972641946 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f064406c8f0 0x7f064406edb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:55.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.460+0000 7f065de5c640 1 --2- 172.21.15.44:0/3972641946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f065815e610 0x7f065814a850 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:55.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.460+0000 7f065de5c640 1 --2- 172.21.15.44:0/3972641946 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0658155340 0x7f06581f0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:55.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.460+0000 7f065de5c640 1 --2- 172.21.15.44:0/3972641946 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f06580cc310 0x7f065814f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:55.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.460+0000 7f065de5c640 1 -- 172.21.15.44:0/3972641946 >> 172.21.15.44:0/3972641946 conn(0x7f065806c560 msgr2=0x7f06580c5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:55.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.460+0000 7f065de5c640 1 -- 172.21.15.44:0/3972641946 shutdown_connections 2024-04-22T06:31:55.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.460+0000 7f065de5c640 1 -- 172.21.15.44:0/3972641946 wait complete. 2024-04-22T06:31:55.469 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T06:31:55.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.572+0000 7fef38d52640 1 Processor -- start 2024-04-22T06:31:55.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.573+0000 7fef38d52640 1 -- start start 2024-04-22T06:31:55.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.573+0000 7fef38d52640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fef34074260 0x7fef3415ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:55.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.573+0000 7fef38d52640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef3406c560 con 0x7fef34154f40 2024-04-22T06:31:55.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.573+0000 7fef38d52640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef3415bfa0 con 0x7fef34155320 2024-04-22T06:31:55.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.573+0000 7fef38d52640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fef3415b420 con 0x7fef34074260 2024-04-22T06:31:55.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.573+0000 7fef3259c640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fef34154f40 0x7fef34073a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:38686/0 (socket says 172.21.15.44:38686) 2024-04-22T06:31:55.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.574+0000 7fef3259c640 1 -- 172.21.15.44:0/2547351906 learned_addr learned my addr 172.21.15.44:0/2547351906 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:55.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.574+0000 7fef32d9d640 1 --2- 172.21.15.44:0/2547351906 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fef34074260 0x7fef3415ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:55.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.574+0000 7fef31d9b640 1 -- 172.21.15.44:0/2547351906 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3243281083 0 0) 0x7fef3415bfa0 con 0x7fef34155320 2024-04-22T06:31:55.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.574+0000 7fef31d9b640 1 -- 172.21.15.44:0/2547351906 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef20003520 con 0x7fef34155320 2024-04-22T06:31:55.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.574+0000 7fef31d9b640 1 -- 172.21.15.44:0/2547351906 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3405826147 0 0) 0x7fef3406c560 con 0x7fef34154f40 2024-04-22T06:31:55.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.574+0000 7fef31d9b640 1 -- 172.21.15.44:0/2547351906 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef3415bfa0 con 0x7fef34154f40 2024-04-22T06:31:55.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.574+0000 7fef32d9d640 1 -- 172.21.15.44:0/2547351906 >> v1:172.21.15.70:6789/0 conn(0x7fef34155320 legacy=0x7fef340ce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:55.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.574+0000 7fef32d9d640 1 -- 172.21.15.44:0/2547351906 >> v1:172.21.15.44:6789/0 conn(0x7fef34154f40 legacy=0x7fef34073a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:55.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.574+0000 7fef32d9d640 1 -- 172.21.15.44:0/2547351906 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef3415b700 con 0x7fef34074260 2024-04-22T06:31:55.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.575+0000 7fef32d9d640 1 --2- 172.21.15.44:0/2547351906 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fef34074260 0x7fef3415ec00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fef2c009dc0 tx=0x7fef2c02bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f25c9ba8727002f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:55.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.575+0000 7fef31d9b640 1 -- 172.21.15.44:0/2547351906 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2c039070 con 0x7fef34074260 2024-04-22T06:31:55.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.575+0000 7fef31d9b640 1 -- 172.21.15.44:0/2547351906 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef2c02cd50 con 0x7fef34074260 2024-04-22T06:31:55.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.575+0000 7fef31d9b640 1 -- 172.21.15.44:0/2547351906 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2c034660 con 0x7fef34074260 2024-04-22T06:31:55.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.575+0000 7fef38d52640 1 -- 172.21.15.44:0/2547351906 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fef34074260 msgr2=0x7fef3415ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:55.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.575+0000 7fef38d52640 1 --2- 172.21.15.44:0/2547351906 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fef34074260 0x7fef3415ec00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fef2c009dc0 tx=0x7fef2c02bfa0 comp rx=0 tx=0).stop 2024-04-22T06:31:55.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.575+0000 7fef38d52640 1 -- 172.21.15.44:0/2547351906 shutdown_connections 2024-04-22T06:31:55.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.575+0000 7fef38d52640 1 --2- 172.21.15.44:0/2547351906 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fef34074260 0x7fef3415ec00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:55.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.575+0000 7fef38d52640 1 -- 172.21.15.44:0/2547351906 >> 172.21.15.44:0/2547351906 conn(0x7fef34074cb0 msgr2=0x7fef340750c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:55.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.575+0000 7fef38d52640 1 -- 172.21.15.44:0/2547351906 shutdown_connections 2024-04-22T06:31:55.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.576+0000 7fef38d52640 1 -- 172.21.15.44:0/2547351906 wait complete. 2024-04-22T06:31:55.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.577+0000 7fef38d52640 1 Processor -- start 2024-04-22T06:31:55.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.577+0000 7fef38d52640 1 -- start start 2024-04-22T06:31:55.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.577+0000 7fef38d52640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fef34074260 0x7fef3415dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:55.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.577+0000 7fef38d52640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef34154f40 0x7fef340cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:55.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.577+0000 7fef38d52640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef34157930 0x7fef340ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:55.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.577+0000 7fef38d52640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fef3415e6e0 con 0x7fef34157930 2024-04-22T06:31:55.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.577+0000 7fef38d52640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fef3415bcc0 con 0x7fef34154f40 2024-04-22T06:31:55.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.577+0000 7fef38d52640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fef3415b050 con 0x7fef34074260 2024-04-22T06:31:55.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.578+0000 7fef32d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef34157930 0x7fef340ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:55.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.578+0000 7fef32d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef34157930 0x7fef340ccae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55112/0 (socket says 172.21.15.44:55112) 2024-04-22T06:31:55.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.578+0000 7fef32d9d640 1 -- 172.21.15.44:0/2455926871 learned_addr learned my addr 172.21.15.44:0/2455926871 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:55.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.578+0000 7fef3259c640 1 --2- 172.21.15.44:0/2455926871 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fef34074260 0x7fef3415dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:55.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.578+0000 7fef2bfff640 1 --2- 172.21.15.44:0/2455926871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef34154f40 0x7fef340cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:55.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.578+0000 7fef32d9d640 1 -- 172.21.15.44:0/2455926871 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fef34074260 msgr2=0x7fef3415dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:55.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.578+0000 7fef32d9d640 1 --2- 172.21.15.44:0/2455926871 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fef34074260 0x7fef3415dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:55.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.578+0000 7fef32d9d640 1 -- 172.21.15.44:0/2455926871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef34154f40 msgr2=0x7fef340cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:55.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.578+0000 7fef32d9d640 1 --2- 172.21.15.44:0/2455926871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef34154f40 0x7fef340cc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:55.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.578+0000 7fef32d9d640 1 -- 172.21.15.44:0/2455926871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef341f0cd0 con 0x7fef34157930 2024-04-22T06:31:55.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.578+0000 7fef32d9d640 1 --2- 172.21.15.44:0/2455926871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef34157930 0x7fef340ccae0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fef2c009ee0 tx=0x7fef2c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:55.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.578+0000 7fef2affd640 1 -- 172.21.15.44:0/2455926871 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2c039050 con 0x7fef34157930 2024-04-22T06:31:55.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.578+0000 7fef2affd640 1 -- 172.21.15.44:0/2455926871 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef2c045070 con 0x7fef34157930 2024-04-22T06:31:55.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.578+0000 7fef38d52640 1 -- 172.21.15.44:0/2455926871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef34159e90 con 0x7fef34157930 2024-04-22T06:31:55.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.578+0000 7fef2affd640 1 -- 172.21.15.44:0/2455926871 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2c009450 con 0x7fef34157930 2024-04-22T06:31:55.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.578+0000 7fef38d52640 1 -- 172.21.15.44:0/2455926871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef341f0f90 con 0x7fef34157930 2024-04-22T06:31:55.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.579+0000 7fef38d52640 1 -- 172.21.15.44:0/2455926871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef34069310 con 0x7fef34157930 2024-04-22T06:31:55.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.580+0000 7fef2affd640 1 -- 172.21.15.44:0/2455926871 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fef2c03e850 con 0x7fef34157930 2024-04-22T06:31:55.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.580+0000 7fef2affd640 1 --2- 172.21.15.44:0/2455926871 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fef1406c8f0 0x7fef1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:55.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.580+0000 7fef2affd640 1 -- 172.21.15.44:0/2455926871 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fef2c0aed70 con 0x7fef34157930 2024-04-22T06:31:55.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.580+0000 7fef3259c640 1 --2- 172.21.15.44:0/2455926871 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fef1406c8f0 0x7fef1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:55.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.581+0000 7fef3259c640 1 --2- 172.21.15.44:0/2455926871 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fef1406c8f0 0x7fef1406edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fef24005880 tx=0x7fef24009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:55.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.584+0000 7fef2affd640 1 -- 172.21.15.44:0/2455926871 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fef2c07c6c0 con 0x7fef34157930 2024-04-22T06:31:55.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.771+0000 7fef38d52640 1 -- 172.21.15.44:0/2455926871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fef34154760 con 0x7fef34157930 2024-04-22T06:31:55.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.772+0000 7fef2affd640 1 -- 172.21.15.44:0/2455926871 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18794 (secure 0 0 0) 0x7fef2c0809c0 con 0x7fef34157930 2024-04-22T06:31:55.771 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:31:55.772 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":27,"fsid":"9b0a34ca-403d-4513-8a66-d3749ba559aa","created":"2024-04-22T06:30:42.334581+0000","modified":"2024-04-22T06:31:48.138913+0000","last_up_change":"2024-04-22T06:30:52.942257+0000","last_in_change":"2024-04-22T06:30:44.467727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T06:30:54.560256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T06:31:04.612387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T06:31:05.312124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-22T06:31:09.025550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b4a6692-53c9-472d-b2f3-1dd29a6679ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6817","nonce":2220268493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6819","nonce":2220268493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6823","nonce":2220268493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6821","nonce":2220268493}]},"public_addr":"172.21.15.44:6817/2220268493","cluster_addr":"172.21.15.44:6819/2220268493","heartbeat_back_addr":"172.21.15.44:6823/2220268493","heartbeat_front_addr":"172.21.15.44:6821/2220268493","state":["exists","up"]},{"osd":1,"uuid":"70a6370a-7c0a-4d64-a0e4-b2e726b4f291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6809","nonce":1278678977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6811","nonce":1278678977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6815","nonce":1278678977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6813","nonce":1278678977}]},"public_addr":"172.21.15.44:6809/1278678977","cluster_addr":"172.21.15.44:6811/1278678977","heartbeat_back_addr":"172.21.15.44:6815/1278678977","heartbeat_front_addr":"172.21.15.44:6813/1278678977","state":["exists","up"]},{"osd":2,"uuid":"f2e51a62-ca91-470d-bd5a-5710c61deee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6801","nonce":2544087903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6803","nonce":2544087903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6807","nonce":2544087903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6805","nonce":2544087903}]},"public_addr":"172.21.15.44:6801/2544087903","cluster_addr":"172.21.15.44:6803/2544087903","heartbeat_back_addr":"172.21.15.44:6807/2544087903","heartbeat_front_addr":"172.21.15.44:6805/2544087903","state":["exists","up"]},{"osd":3,"uuid":"bc514203-9af9-4f1f-b18e-765bdd5de6e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6825","nonce":1692928109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6827","nonce":1692928109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6831","nonce":1692928109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6829","nonce":1692928109}]},"public_addr":"172.21.15.44:6825/1692928109","cluster_addr":"172.21.15.44:6827/1692928109","heartbeat_back_addr":"172.21.15.44:6831/1692928109","heartbeat_front_addr":"172.21.15.44:6829/1692928109","state":["exists","up"]},{"osd":4,"uuid":"357c439a-6e08-4c19-ac18-b68e6c890497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6820","nonce":3802577427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6824","nonce":3802577427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6831","nonce":3802577427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6828","nonce":3802577427}]},"public_addr":"172.21.15.70:6820/3802577427","cluster_addr":"172.21.15.70:6824/3802577427","heartbeat_back_addr":"172.21.15.70:6831/3802577427","heartbeat_front_addr":"172.21.15.70:6828/3802577427","state":["exists","up"]},{"osd":5,"uuid":"952fb88f-d56e-4305-bfa8-d32674332dfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6801","nonce":664757345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6803","nonce":664757345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6807","nonce":664757345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6805","nonce":664757345}]},"public_addr":"172.21.15.70:6801/664757345","cluster_addr":"172.21.15.70:6803/664757345","heartbeat_back_addr":"172.21.15.70:6807/664757345","heartbeat_front_addr":"172.21.15.70:6805/664757345","state":["exists","up"]},{"osd":6,"uuid":"74399b3d-fac2-45ab-9172-2cd95b02450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6817","nonce":1623253482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6821","nonce":1623253482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6827","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6829","nonce":1623253482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6825","nonce":1623253482}]},"public_addr":"172.21.15.70:6817/1623253482","cluster_addr":"172.21.15.70:6821/1623253482","heartbeat_back_addr":"172.21.15.70:6829/1623253482","heartbeat_front_addr":"172.21.15.70:6825/1623253482","state":["exists","up"]},{"osd":7,"uuid":"8f35508d-6fcc-43ff-8ee7-0498a7ab3b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6809","nonce":1551180300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6811","nonce":1551180300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6815","nonce":1551180300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6813","nonce":1551180300}]},"public_addr":"172.21.15.70:6809/1551180300","cluster_addr":"172.21.15.70:6811/1551180300","heartbeat_back_addr":"172.21.15.70:6815/1551180300","heartbeat_front_addr":"172.21.15.70:6813/1551180300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.612502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.653756+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.521169+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.649955+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.640231+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.686638+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.642199+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.700360+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T06:31:55.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.774+0000 7fef38d52640 1 -- 172.21.15.44:0/2455926871 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fef1406c8f0 msgr2=0x7fef1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:55.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.774+0000 7fef38d52640 1 --2- 172.21.15.44:0/2455926871 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fef1406c8f0 0x7fef1406edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fef24005880 tx=0x7fef24009040 comp rx=0 tx=0).stop 2024-04-22T06:31:55.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.774+0000 7fef38d52640 1 -- 172.21.15.44:0/2455926871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef34157930 msgr2=0x7fef340ccae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:55.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.774+0000 7fef38d52640 1 --2- 172.21.15.44:0/2455926871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef34157930 0x7fef340ccae0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fef2c009ee0 tx=0x7fef2c004a40 comp rx=0 tx=0).stop 2024-04-22T06:31:55.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.774+0000 7fef38d52640 1 -- 172.21.15.44:0/2455926871 shutdown_connections 2024-04-22T06:31:55.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.774+0000 7fef38d52640 1 --2- 172.21.15.44:0/2455926871 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fef1406c8f0 0x7fef1406edb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:55.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.774+0000 7fef38d52640 1 --2- 172.21.15.44:0/2455926871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef34157930 0x7fef340ccae0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:55.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.774+0000 7fef38d52640 1 --2- 172.21.15.44:0/2455926871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef34154f40 0x7fef340cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:55.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.774+0000 7fef38d52640 1 --2- 172.21.15.44:0/2455926871 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fef34074260 0x7fef3415dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:55.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.774+0000 7fef38d52640 1 -- 172.21.15.44:0/2455926871 >> 172.21.15.44:0/2455926871 conn(0x7fef34074cb0 msgr2=0x7fef34158ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:55.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.775+0000 7fef38d52640 1 -- 172.21.15.44:0/2455926871 shutdown_connections 2024-04-22T06:31:55.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.775+0000 7fef38d52640 1 -- 172.21.15.44:0/2455926871 wait complete. 2024-04-22T06:31:55.785 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T06:31:55.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.886+0000 7f5ef6ce9640 1 Processor -- start 2024-04-22T06:31:55.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.886+0000 7f5ef6ce9640 1 -- start start 2024-04-22T06:31:55.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.887+0000 7f5ef6ce9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5ef015f0d0 0x7f5ef00d0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:55.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.887+0000 7f5ef6ce9640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ef015b200 con 0x7f5ef00d1400 2024-04-22T06:31:55.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.887+0000 7f5ef6ce9640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ef015b4e0 con 0x7f5ef0155240 2024-04-22T06:31:55.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.887+0000 7f5ef6ce9640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f5ef015c060 con 0x7f5ef015f0d0 2024-04-22T06:31:55.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.887+0000 7f5ef5286640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f5ef00d1400 0x7f5ef00cc480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:38692/0 (socket says 172.21.15.44:38692) 2024-04-22T06:31:55.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.887+0000 7f5ef5286640 1 -- 172.21.15.44:0/1501142350 learned_addr learned my addr 172.21.15.44:0/1501142350 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:55.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.887+0000 7f5eeffff640 1 --2- 172.21.15.44:0/1501142350 >> v2:172.21.15.70:3300/0 conn(0x7f5ef015f0d0 0x7f5ef00d0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:55.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.887+0000 7f5eef7fe640 1 -- 172.21.15.44:0/1501142350 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1276708847 0 0) 0x7f5ef015b200 con 0x7f5ef00d1400 2024-04-22T06:31:55.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.888+0000 7f5eef7fe640 1 -- 172.21.15.44:0/1501142350 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5edc003520 con 0x7f5ef00d1400 2024-04-22T06:31:55.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.888+0000 7f5eef7fe640 1 -- 172.21.15.44:0/1501142350 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1526516634 0 0) 0x7f5ef015b4e0 con 0x7f5ef0155240 2024-04-22T06:31:55.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.888+0000 7f5eef7fe640 1 -- 172.21.15.44:0/1501142350 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ef015b200 con 0x7f5ef0155240 2024-04-22T06:31:55.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.888+0000 7f5eeffff640 1 -- 172.21.15.44:0/1501142350 >> v1:172.21.15.70:6789/0 conn(0x7f5ef0155240 legacy=0x7f5ef00cb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:55.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.888+0000 7f5eeffff640 1 -- 172.21.15.44:0/1501142350 >> v1:172.21.15.44:6789/0 conn(0x7f5ef00d1400 legacy=0x7f5ef00cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:55.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.888+0000 7f5eeffff640 1 -- 172.21.15.44:0/1501142350 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef015b7c0 con 0x7f5ef015f0d0 2024-04-22T06:31:55.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.888+0000 7f5eeffff640 1 --2- 172.21.15.44:0/1501142350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ef015f0d0 0x7f5ef00d0ec0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5ed80097a0 tx=0x7f5ed802b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e19ebecbf38902e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:55.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.889+0000 7f5eef7fe640 1 -- 172.21.15.44:0/1501142350 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed8034070 con 0x7f5ef015f0d0 2024-04-22T06:31:55.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.889+0000 7f5eef7fe640 1 -- 172.21.15.44:0/1501142350 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ed8030610 con 0x7f5ef015f0d0 2024-04-22T06:31:55.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.889+0000 7f5eef7fe640 1 -- 172.21.15.44:0/1501142350 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed80309e0 con 0x7f5ef015f0d0 2024-04-22T06:31:55.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.889+0000 7f5ef6ce9640 1 -- 172.21.15.44:0/1501142350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ef015f0d0 msgr2=0x7f5ef00d0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:55.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.889+0000 7f5ef6ce9640 1 --2- 172.21.15.44:0/1501142350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ef015f0d0 0x7f5ef00d0ec0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5ed80097a0 tx=0x7f5ed802b960 comp rx=0 tx=0).stop 2024-04-22T06:31:55.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.889+0000 7f5ef6ce9640 1 -- 172.21.15.44:0/1501142350 shutdown_connections 2024-04-22T06:31:55.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.889+0000 7f5ef6ce9640 1 --2- 172.21.15.44:0/1501142350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ef015f0d0 0x7f5ef00d0ec0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:55.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.889+0000 7f5ef6ce9640 1 -- 172.21.15.44:0/1501142350 >> 172.21.15.44:0/1501142350 conn(0x7f5ef00750d0 msgr2=0x7f5ef00754e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:55.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.889+0000 7f5ef6ce9640 1 -- 172.21.15.44:0/1501142350 shutdown_connections 2024-04-22T06:31:55.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.889+0000 7f5ef6ce9640 1 -- 172.21.15.44:0/1501142350 wait complete. 2024-04-22T06:31:55.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.890+0000 7f5ef6ce9640 1 Processor -- start 2024-04-22T06:31:55.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.890+0000 7f5ef6ce9640 1 -- start start 2024-04-22T06:31:55.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.891+0000 7f5ef6ce9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ef00d1400 0x7f5ef014bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:55.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.891+0000 7f5ef6ce9640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5ef0155240 0x7f5ef00c9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:55.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.891+0000 7f5ef6ce9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ef015f0d0 0x7f5ef01f1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:55.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.891+0000 7f5ef6ce9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef015f520 con 0x7f5ef015f0d0 2024-04-22T06:31:55.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.891+0000 7f5ef6ce9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef015bd80 con 0x7f5ef00d1400 2024-04-22T06:31:55.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.891+0000 7f5ef6ce9640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f5ef0155620 con 0x7f5ef0155240 2024-04-22T06:31:55.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.891+0000 7f5ef5286640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ef015f0d0 0x7f5ef01f1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:55.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.891+0000 7f5ef5286640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ef015f0d0 0x7f5ef01f1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55120/0 (socket says 172.21.15.44:55120) 2024-04-22T06:31:55.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.891+0000 7f5ef5286640 1 -- 172.21.15.44:0/2520308100 learned_addr learned my addr 172.21.15.44:0/2520308100 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:55.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.891+0000 7f5ef4a85640 1 --2- 172.21.15.44:0/2520308100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ef00d1400 0x7f5ef014bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:55.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.891+0000 7f5eeffff640 1 --2- 172.21.15.44:0/2520308100 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5ef0155240 0x7f5ef00c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:55.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.892+0000 7f5ef5286640 1 -- 172.21.15.44:0/2520308100 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5ef0155240 msgr2=0x7f5ef00c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:55.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.892+0000 7f5ef5286640 1 --2- 172.21.15.44:0/2520308100 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5ef0155240 0x7f5ef00c9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:55.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.892+0000 7f5ef5286640 1 -- 172.21.15.44:0/2520308100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ef00d1400 msgr2=0x7f5ef014bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:55.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.892+0000 7f5ef5286640 1 --2- 172.21.15.44:0/2520308100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ef00d1400 0x7f5ef014bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:55.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.892+0000 7f5ef5286640 1 -- 172.21.15.44:0/2520308100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef00c7de0 con 0x7f5ef015f0d0 2024-04-22T06:31:55.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.892+0000 7f5eeffff640 1 --2- 172.21.15.44:0/2520308100 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5ef0155240 0x7f5ef00c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:31:55.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.892+0000 7f5ef5286640 1 --2- 172.21.15.44:0/2520308100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ef015f0d0 0x7f5ef01f1170 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5ee000c430 tx=0x7f5ee000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:55.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.892+0000 7f5eed7fa640 1 -- 172.21.15.44:0/2520308100 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0017070 con 0x7f5ef015f0d0 2024-04-22T06:31:55.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.892+0000 7f5eed7fa640 1 -- 172.21.15.44:0/2520308100 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ee0005ce0 con 0x7f5ef015f0d0 2024-04-22T06:31:55.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.892+0000 7f5ef6ce9640 1 -- 172.21.15.44:0/2520308100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ef00c8080 con 0x7f5ef015f0d0 2024-04-22T06:31:55.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.892+0000 7f5eed7fa640 1 -- 172.21.15.44:0/2520308100 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee00072f0 con 0x7f5ef015f0d0 2024-04-22T06:31:55.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.892+0000 7f5ef6ce9640 1 -- 172.21.15.44:0/2520308100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ef00c8600 con 0x7f5ef015f0d0 2024-04-22T06:31:55.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.893+0000 7f5ef6ce9640 1 -- 172.21.15.44:0/2520308100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ef0068cc0 con 0x7f5ef015f0d0 2024-04-22T06:31:55.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.894+0000 7f5eed7fa640 1 -- 172.21.15.44:0/2520308100 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ee001a070 con 0x7f5ef015f0d0 2024-04-22T06:31:55.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.894+0000 7f5eed7fa640 1 --2- 172.21.15.44:0/2520308100 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f5ed006c8f0 0x7f5ed006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:55.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.894+0000 7f5eed7fa640 1 -- 172.21.15.44:0/2520308100 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f5ee008b330 con 0x7f5ef015f0d0 2024-04-22T06:31:55.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.894+0000 7f5ef4a85640 1 --2- 172.21.15.44:0/2520308100 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f5ed006c8f0 0x7f5ed006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:55.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.895+0000 7f5ef4a85640 1 --2- 172.21.15.44:0/2520308100 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f5ed006c8f0 0x7f5ed006edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8001ea0 tx=0x7f5ee8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:55.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:55.898+0000 7f5eed7fa640 1 -- 172.21.15.44:0/2520308100 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5ee0058d10 con 0x7f5ef015f0d0 2024-04-22T06:31:56.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:56.125+0000 7f5ef6ce9640 1 -- 172.21.15.44:0/2520308100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ef00b3ee0 con 0x7f5ef015f0d0 2024-04-22T06:31:56.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:56.126+0000 7f5eed7fa640 1 -- 172.21.15.44:0/2520308100 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6901 (secure 0 0 0) 0x7f5ee005d010 con 0x7f5ef015f0d0 2024-04-22T06:31:56.125 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:31:56.125 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/1779271860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":1779271860},{"type":"v1","addr":"172.21.15.70:6836","nonce":1779271860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T06:31:06.356312+0000","modified":"2024-04-22T06:31:50.151090+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4308,"mds_1":4305,"mds_2":4293,"mds_3":4290,"mds_4":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":4,"incarnation":17,"state":"up:creating","state_seq":1,"addr":"172.21.15.44:6834/1497302315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1497302315},{"type":"v1","addr":"172.21.15.44:6834","nonce":1497302315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4290":{"gid":4290,"name":"a","rank":3,"incarnation":15,"state":"up:active","state_seq":14,"addr":"172.21.15.44:6835/2383621014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2383621014},{"type":"v1","addr":"172.21.15.44:6835","nonce":2383621014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"e","rank":2,"incarnation":13,"state":"up:active","state_seq":12,"addr":"172.21.15.44:6837/915960587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":915960587},{"type":"v1","addr":"172.21.15.44:6837","nonce":915960587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4305":{"gid":4305,"name":"b","rank":1,"incarnation":9,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6838/924955785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":924955785},{"type":"v1","addr":"172.21.15.70:6838","nonce":924955785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4308":{"gid":4308,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.70:6839/2915150040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6837","nonce":2915150040},{"type":"v1","addr":"172.21.15.70:6839","nonce":2915150040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4308,"qdb_cluster":[4308,4305,4293,4290]},"id":1}]} 2024-04-22T06:31:56.125 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 17 2024-04-22T06:31:56.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:56.128+0000 7f5ef6ce9640 1 -- 172.21.15.44:0/2520308100 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f5ed006c8f0 msgr2=0x7f5ed006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:56.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:56.128+0000 7f5ef6ce9640 1 --2- 172.21.15.44:0/2520308100 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f5ed006c8f0 0x7f5ed006edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8001ea0 tx=0x7f5ee8009040 comp rx=0 tx=0).stop 2024-04-22T06:31:56.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:56.128+0000 7f5ef6ce9640 1 -- 172.21.15.44:0/2520308100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ef015f0d0 msgr2=0x7f5ef01f1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:56.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:56.128+0000 7f5ef6ce9640 1 --2- 172.21.15.44:0/2520308100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ef015f0d0 0x7f5ef01f1170 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5ee000c430 tx=0x7f5ee000c8f0 comp rx=0 tx=0).stop 2024-04-22T06:31:56.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:56.128+0000 7f5ef6ce9640 1 -- 172.21.15.44:0/2520308100 shutdown_connections 2024-04-22T06:31:56.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:56.128+0000 7f5ef6ce9640 1 --2- 172.21.15.44:0/2520308100 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f5ed006c8f0 0x7f5ed006edb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:56.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:56.128+0000 7f5ef6ce9640 1 --2- 172.21.15.44:0/2520308100 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5ef0155240 0x7f5ef00c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:56.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:56.128+0000 7f5ef6ce9640 1 --2- 172.21.15.44:0/2520308100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ef015f0d0 0x7f5ef01f1170 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:56.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:56.128+0000 7f5ef6ce9640 1 --2- 172.21.15.44:0/2520308100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ef00d1400 0x7f5ef014bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:56.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:56.128+0000 7f5ef6ce9640 1 -- 172.21.15.44:0/2520308100 >> 172.21.15.44:0/2520308100 conn(0x7f5ef00750d0 msgr2=0x7f5ef0153400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:56.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:56.128+0000 7f5ef6ce9640 1 -- 172.21.15.44:0/2520308100 shutdown_connections 2024-04-22T06:31:56.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:56.128+0000 7f5ef6ce9640 1 -- 172.21.15.44:0/2520308100 wait complete. 2024-04-22T06:31:56.744 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-22T06:31:06.356312+0000', 'modified': '2024-04-22T06:31:50.151090+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 4308, 'mds_1': 4305, 'mds_2': 4293, 'mds_3': 4290, 'mds_4': 4287}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4287': {'gid': 4287, 'name': 'c', 'rank': 4, 'incarnation': 17, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.44:6834/1497302315', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6832', 'nonce': 1497302315}, {'type': 'v1', 'addr': '172.21.15.44:6834', 'nonce': 1497302315}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4290': {'gid': 4290, 'name': 'a', 'rank': 3, 'incarnation': 15, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.44:6835/2383621014', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6833', 'nonce': 2383621014}, {'type': 'v1', 'addr': '172.21.15.44:6835', 'nonce': 2383621014}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4293': {'gid': 4293, 'name': 'e', 'rank': 2, 'incarnation': 13, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.44:6837/915960587', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6836', 'nonce': 915960587}, {'type': 'v1', 'addr': '172.21.15.44:6837', 'nonce': 915960587}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4305': {'gid': 4305, 'name': 'b', 'rank': 1, 'incarnation': 9, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.70:6838/924955785', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6834', 'nonce': 924955785}, {'type': 'v1', 'addr': '172.21.15.70:6838', 'nonce': 924955785}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4308': {'gid': 4308, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.70:6839/2915150040', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6837', 'nonce': 2915150040}, {'type': 'v1', 'addr': '172.21.15.70:6839', 'nonce': 2915150040}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4308, 'qdb_cluster': [4308, 4305, 4293, 4290]} 2024-04-22T06:31:56.744 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_4287:up:creating 2024-04-22T06:31:57.746 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T06:31:57.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.861+0000 7fa0aa750640 1 Processor -- start 2024-04-22T06:31:57.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.861+0000 7fa0aa750640 1 -- start start 2024-04-22T06:31:57.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.861+0000 7fa0aa750640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fa0a41556d0 0x7fa0a415b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:57.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.861+0000 7fa0aa750640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0a415e770 0x7fa0a40c9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:57.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.861+0000 7fa0aa750640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa0a40c7d00 0x7fa0a40c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:57.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.861+0000 7fa0aa750640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fa0a40c8910 con 0x7fa0a41556d0 2024-04-22T06:31:57.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.861+0000 7fa0aa750640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fa0a40c8ed0 con 0x7fa0a40c7d00 2024-04-22T06:31:57.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.861+0000 7fa0aa750640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa0a415b740 con 0x7fa0a415e770 2024-04-22T06:31:57.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.862+0000 7fa0a37fe640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0a415e770 0x7fa0a40c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:57.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.862+0000 7fa0a8ced640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa0a40c7d00 0x7fa0a40c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:57.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.862+0000 7fa0a3fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fa0a41556d0 0x7fa0a415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:57.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.862+0000 7fa0a37fe640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0a415e770 0x7fa0a40c9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.44:45596/0 (socket says 172.21.15.44:45596) 2024-04-22T06:31:57.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.862+0000 7fa0a37fe640 1 -- 172.21.15.44:0/1420569714 learned_addr learned my addr 172.21.15.44:0/1420569714 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:57.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.862+0000 7fa0a37fe640 1 -- 172.21.15.44:0/1420569714 >> v2:172.21.15.70:3300/0 conn(0x7fa0a40c7d00 msgr2=0x7fa0a40c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:57.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.862+0000 7fa0a37fe640 1 --2- 172.21.15.44:0/1420569714 >> v2:172.21.15.70:3300/0 conn(0x7fa0a40c7d00 0x7fa0a40c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:57.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.862+0000 7fa0a37fe640 1 -- 172.21.15.44:0/1420569714 >> v2:172.21.15.44:3300/0 conn(0x7fa0a41556d0 msgr2=0x7fa0a415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:57.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.862+0000 7fa0a37fe640 1 --2- 172.21.15.44:0/1420569714 >> v2:172.21.15.44:3300/0 conn(0x7fa0a41556d0 0x7fa0a415b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:57.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.862+0000 7fa0a37fe640 1 -- 172.21.15.44:0/1420569714 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a40b6870 con 0x7fa0a415e770 2024-04-22T06:31:57.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.863+0000 7fa0a37fe640 1 --2- 172.21.15.44:0/1420569714 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0a415e770 0x7fa0a40c9360 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa0a40ca690 tx=0x7fa09402b9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae2350a6688015a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:57.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.863+0000 7fa0a2ffd640 1 -- 172.21.15.44:0/1420569714 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa094034070 con 0x7fa0a415e770 2024-04-22T06:31:57.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.863+0000 7fa0a2ffd640 1 -- 172.21.15.44:0/1420569714 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa094030860 con 0x7fa0a415e770 2024-04-22T06:31:57.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.863+0000 7fa0a2ffd640 1 -- 172.21.15.44:0/1420569714 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa094030c30 con 0x7fa0a415e770 2024-04-22T06:31:57.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.863+0000 7fa0aa750640 1 -- 172.21.15.44:0/1420569714 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0a415e770 msgr2=0x7fa0a40c9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:57.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.863+0000 7fa0aa750640 1 --2- 172.21.15.44:0/1420569714 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0a415e770 0x7fa0a40c9360 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa0a40ca690 tx=0x7fa09402b9a0 comp rx=0 tx=0).stop 2024-04-22T06:31:57.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.863+0000 7fa0aa750640 1 -- 172.21.15.44:0/1420569714 shutdown_connections 2024-04-22T06:31:57.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.863+0000 7fa0aa750640 1 --2- 172.21.15.44:0/1420569714 >> v2:172.21.15.70:3300/0 conn(0x7fa0a40c7d00 0x7fa0a40c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:57.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.863+0000 7fa0aa750640 1 --2- 172.21.15.44:0/1420569714 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0a415e770 0x7fa0a40c9360 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:57.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.863+0000 7fa0aa750640 1 --2- 172.21.15.44:0/1420569714 >> v2:172.21.15.44:3300/0 conn(0x7fa0a41556d0 0x7fa0a415b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:57.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.863+0000 7fa0aa750640 1 -- 172.21.15.44:0/1420569714 >> 172.21.15.44:0/1420569714 conn(0x7fa0a4072fc0 msgr2=0x7fa0a40733d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:57.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.863+0000 7fa0aa750640 1 -- 172.21.15.44:0/1420569714 shutdown_connections 2024-04-22T06:31:57.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.864+0000 7fa0aa750640 1 -- 172.21.15.44:0/1420569714 wait complete. 2024-04-22T06:31:57.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.865+0000 7fa0aa750640 1 Processor -- start 2024-04-22T06:31:57.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.865+0000 7fa0aa750640 1 -- start start 2024-04-22T06:31:57.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.866+0000 7fa0aa750640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0a40c7d00 0x7fa0a40cb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:57.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.866+0000 7fa0aa750640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0a41556d0 0x7fa0a41f4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:57.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.866+0000 7fa0aa750640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0a415e770 0x7fa0a4157c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:57.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.866+0000 7fa0aa750640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a415bf90 con 0x7fa0a41556d0 2024-04-22T06:31:57.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.866+0000 7fa0aa750640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a415bcb0 con 0x7fa0a415e770 2024-04-22T06:31:57.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.866+0000 7fa0aa750640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa0a40c8b60 con 0x7fa0a40c7d00 2024-04-22T06:31:57.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.866+0000 7fa0a37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0a41556d0 0x7fa0a41f4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:57.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.866+0000 7fa0a37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0a41556d0 0x7fa0a41f4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55146/0 (socket says 172.21.15.44:55146) 2024-04-22T06:31:57.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.866+0000 7fa0a37fe640 1 -- 172.21.15.44:0/3180151880 learned_addr learned my addr 172.21.15.44:0/3180151880 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:57.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.866+0000 7fa0a3fff640 1 --2- 172.21.15.44:0/3180151880 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0a40c7d00 0x7fa0a40cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:57.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.866+0000 7fa0a8ced640 1 --2- 172.21.15.44:0/3180151880 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0a415e770 0x7fa0a4157c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:57.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.866+0000 7fa0a37fe640 1 -- 172.21.15.44:0/3180151880 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0a40c7d00 msgr2=0x7fa0a40cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:57.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.866+0000 7fa0a37fe640 1 --2- 172.21.15.44:0/3180151880 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0a40c7d00 0x7fa0a40cb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:57.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.866+0000 7fa0a37fe640 1 -- 172.21.15.44:0/3180151880 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0a415e770 msgr2=0x7fa0a4157c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:57.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.866+0000 7fa0a37fe640 1 --2- 172.21.15.44:0/3180151880 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0a415e770 0x7fa0a4157c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:57.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.866+0000 7fa0a37fe640 1 -- 172.21.15.44:0/3180151880 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a41f4f30 con 0x7fa0a41556d0 2024-04-22T06:31:57.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.866+0000 7fa0a37fe640 1 --2- 172.21.15.44:0/3180151880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0a41556d0 0x7fa0a41f4670 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa094000c00 tx=0x7fa0940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:57.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.866+0000 7fa0a0ff9640 1 -- 172.21.15.44:0/3180151880 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa094034040 con 0x7fa0a41556d0 2024-04-22T06:31:57.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.866+0000 7fa0a0ff9640 1 -- 172.21.15.44:0/3180151880 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa094039d90 con 0x7fa0a41556d0 2024-04-22T06:31:57.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.866+0000 7fa0aa750640 1 -- 172.21.15.44:0/3180151880 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0a41f51a0 con 0x7fa0a41556d0 2024-04-22T06:31:57.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.866+0000 7fa0a0ff9640 1 -- 172.21.15.44:0/3180151880 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0940433f0 con 0x7fa0a41556d0 2024-04-22T06:31:57.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.866+0000 7fa0aa750640 1 -- 172.21.15.44:0/3180151880 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0a4152e70 con 0x7fa0a41556d0 2024-04-22T06:31:57.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.867+0000 7fa0aa750640 1 -- 172.21.15.44:0/3180151880 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0a4069310 con 0x7fa0a41556d0 2024-04-22T06:31:57.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.868+0000 7fa0a0ff9640 1 -- 172.21.15.44:0/3180151880 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa094037070 con 0x7fa0a41556d0 2024-04-22T06:31:57.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.868+0000 7fa0a0ff9640 1 --2- 172.21.15.44:0/3180151880 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fa08406c8f0 0x7fa08406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:57.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.868+0000 7fa0a0ff9640 1 -- 172.21.15.44:0/3180151880 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa0940adea0 con 0x7fa0a41556d0 2024-04-22T06:31:57.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.869+0000 7fa0a3fff640 1 --2- 172.21.15.44:0/3180151880 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fa08406c8f0 0x7fa08406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:57.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.869+0000 7fa0a3fff640 1 --2- 172.21.15.44:0/3180151880 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fa08406c8f0 0x7fa08406edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa08c009ac0 tx=0x7fa08c009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:57.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:57.873+0000 7fa0a0ff9640 1 -- 172.21.15.44:0/3180151880 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa09403e070 con 0x7fa0a41556d0 2024-04-22T06:31:58.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.100+0000 7fa0aa750640 1 -- 172.21.15.44:0/3180151880 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0a41575d0 con 0x7fa0a41556d0 2024-04-22T06:31:58.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.101+0000 7fa0a0ff9640 1 -- 172.21.15.44:0/3180151880 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+6905 (secure 0 0 0) 0x7fa09407b880 con 0x7fa0a41556d0 2024-04-22T06:31:58.100 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:31:58.100 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/1779271860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":1779271860},{"type":"v1","addr":"172.21.15.70:6836","nonce":1779271860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T06:31:06.356312+0000","modified":"2024-04-22T06:31:56.168744+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4308,"mds_1":4305,"mds_2":4293,"mds_3":4290,"mds_4":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":4,"incarnation":17,"state":"up:active","state_seq":15,"addr":"172.21.15.44:6834/1497302315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1497302315},{"type":"v1","addr":"172.21.15.44:6834","nonce":1497302315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4290":{"gid":4290,"name":"a","rank":3,"incarnation":15,"state":"up:active","state_seq":14,"addr":"172.21.15.44:6835/2383621014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2383621014},{"type":"v1","addr":"172.21.15.44:6835","nonce":2383621014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"e","rank":2,"incarnation":13,"state":"up:active","state_seq":12,"addr":"172.21.15.44:6837/915960587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":915960587},{"type":"v1","addr":"172.21.15.44:6837","nonce":915960587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4305":{"gid":4305,"name":"b","rank":1,"incarnation":9,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6838/924955785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":924955785},{"type":"v1","addr":"172.21.15.70:6838","nonce":924955785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4308":{"gid":4308,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.70:6839/2915150040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6837","nonce":2915150040},{"type":"v1","addr":"172.21.15.70:6839","nonce":2915150040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4308,"qdb_cluster":[4287,4290,4293,4305,4308]},"id":1}]} 2024-04-22T06:31:58.100 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 18 2024-04-22T06:31:58.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.103+0000 7fa0aa750640 1 -- 172.21.15.44:0/3180151880 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fa08406c8f0 msgr2=0x7fa08406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:58.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.103+0000 7fa0aa750640 1 --2- 172.21.15.44:0/3180151880 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fa08406c8f0 0x7fa08406edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa08c009ac0 tx=0x7fa08c009190 comp rx=0 tx=0).stop 2024-04-22T06:31:58.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.103+0000 7fa0aa750640 1 -- 172.21.15.44:0/3180151880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0a41556d0 msgr2=0x7fa0a41f4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:58.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.103+0000 7fa0aa750640 1 --2- 172.21.15.44:0/3180151880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0a41556d0 0x7fa0a41f4670 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa094000c00 tx=0x7fa0940046c0 comp rx=0 tx=0).stop 2024-04-22T06:31:58.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.103+0000 7fa0aa750640 1 -- 172.21.15.44:0/3180151880 shutdown_connections 2024-04-22T06:31:58.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.103+0000 7fa0aa750640 1 --2- 172.21.15.44:0/3180151880 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fa08406c8f0 0x7fa08406edb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:58.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.103+0000 7fa0aa750640 1 --2- 172.21.15.44:0/3180151880 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0a415e770 0x7fa0a4157c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:58.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.103+0000 7fa0aa750640 1 --2- 172.21.15.44:0/3180151880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0a41556d0 0x7fa0a41f4670 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:58.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.103+0000 7fa0aa750640 1 --2- 172.21.15.44:0/3180151880 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0a40c7d00 0x7fa0a40cb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:58.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.103+0000 7fa0aa750640 1 -- 172.21.15.44:0/3180151880 >> 172.21.15.44:0/3180151880 conn(0x7fa0a4072fc0 msgr2=0x7fa0a415d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:58.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.103+0000 7fa0aa750640 1 -- 172.21.15.44:0/3180151880 shutdown_connections 2024-04-22T06:31:58.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.103+0000 7fa0aa750640 1 -- 172.21.15.44:0/3180151880 wait complete. 2024-04-22T06:31:58.717 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-22T06:31:06.356312+0000', 'modified': '2024-04-22T06:31:56.168744+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 4308, 'mds_1': 4305, 'mds_2': 4293, 'mds_3': 4290, 'mds_4': 4287}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4287': {'gid': 4287, 'name': 'c', 'rank': 4, 'incarnation': 17, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.44:6834/1497302315', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6832', 'nonce': 1497302315}, {'type': 'v1', 'addr': '172.21.15.44:6834', 'nonce': 1497302315}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4290': {'gid': 4290, 'name': 'a', 'rank': 3, 'incarnation': 15, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.44:6835/2383621014', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6833', 'nonce': 2383621014}, {'type': 'v1', 'addr': '172.21.15.44:6835', 'nonce': 2383621014}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4293': {'gid': 4293, 'name': 'e', 'rank': 2, 'incarnation': 13, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.44:6837/915960587', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6836', 'nonce': 915960587}, {'type': 'v1', 'addr': '172.21.15.44:6837', 'nonce': 915960587}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4305': {'gid': 4305, 'name': 'b', 'rank': 1, 'incarnation': 9, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.70:6838/924955785', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6834', 'nonce': 924955785}, {'type': 'v1', 'addr': '172.21.15.70:6838', 'nonce': 924955785}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4308': {'gid': 4308, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.70:6839/2915150040', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6837', 'nonce': 2915150040}, {'type': 'v1', 'addr': '172.21.15.70:6839', 'nonce': 2915150040}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4308, 'qdb_cluster': [4287, 4290, 4293, 4305, 4308]} 2024-04-22T06:31:58.717 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-22T06:31:58.717 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-22T06:31:58.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.833+0000 7fddf8226640 1 Processor -- start 2024-04-22T06:31:58.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.833+0000 7fddf8226640 1 -- start start 2024-04-22T06:31:58.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.834+0000 7fddf8226640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fddf01552e0 0x7fddf00cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:58.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.834+0000 7fddf8226640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddf015ccf0 con 0x7fddf0154f00 2024-04-22T06:31:58.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.834+0000 7fddf8226640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddf00c5b30 con 0x7fddf006d470 2024-04-22T06:31:58.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.834+0000 7fddf8226640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fddf00c6740 con 0x7fddf01552e0 2024-04-22T06:31:58.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.834+0000 7fddf57c1640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fddf01552e0 0x7fddf00cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:58.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.834+0000 7fddf5fc2640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fddf0154f00 0x7fddf015c5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:38696/0 (socket says 172.21.15.44:38696) 2024-04-22T06:31:58.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.834+0000 7fddf5fc2640 1 -- 172.21.15.44:0/572901171 learned_addr learned my addr 172.21.15.44:0/572901171 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:31:58.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.834+0000 7fddf57c1640 1 -- 172.21.15.44:0/572901171 >> v1:172.21.15.70:6789/0 conn(0x7fddf006d470 legacy=0x7fddf00d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:58.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.834+0000 7fddf57c1640 1 -- 172.21.15.44:0/572901171 >> v1:172.21.15.44:6789/0 conn(0x7fddf0154f00 legacy=0x7fddf015c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:58.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.834+0000 7fddf57c1640 1 -- 172.21.15.44:0/572901171 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf00c5e10 con 0x7fddf01552e0 2024-04-22T06:31:58.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.834+0000 7fddf4fc0640 1 -- 172.21.15.44:0/572901171 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1588781147 0 0) 0x7fddf015ccf0 con 0x7fddf0154f00 2024-04-22T06:31:58.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.835+0000 7fddf57c1640 1 --2- 172.21.15.44:0/572901171 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddf01552e0 0x7fddf00cb430 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdde00097a0 tx=0x7fdde002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7a2bf883d11b50c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:58.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.835+0000 7fddf4fc0640 1 -- 172.21.15.44:0/572901171 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde0035020 con 0x7fddf01552e0 2024-04-22T06:31:58.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.835+0000 7fddf4fc0640 1 -- 172.21.15.44:0/572901171 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdde0031830 con 0x7fddf01552e0 2024-04-22T06:31:58.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.835+0000 7fddf4fc0640 1 -- 172.21.15.44:0/572901171 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde0031c00 con 0x7fddf01552e0 2024-04-22T06:31:58.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.836+0000 7fddf8226640 1 -- 172.21.15.44:0/572901171 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddf01552e0 msgr2=0x7fddf00cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:58.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.836+0000 7fddf8226640 1 --2- 172.21.15.44:0/572901171 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddf01552e0 0x7fddf00cb430 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdde00097a0 tx=0x7fdde002b8d0 comp rx=0 tx=0).stop 2024-04-22T06:31:58.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.836+0000 7fddf8226640 1 -- 172.21.15.44:0/572901171 shutdown_connections 2024-04-22T06:31:58.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.836+0000 7fddf8226640 1 --2- 172.21.15.44:0/572901171 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddf01552e0 0x7fddf00cb430 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:58.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.836+0000 7fddf8226640 1 -- 172.21.15.44:0/572901171 >> 172.21.15.44:0/572901171 conn(0x7fddf00733c0 msgr2=0x7fddf00737d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:58.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.836+0000 7fddf8226640 1 -- 172.21.15.44:0/572901171 shutdown_connections 2024-04-22T06:31:58.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.836+0000 7fddf8226640 1 -- 172.21.15.44:0/572901171 wait complete. 2024-04-22T06:31:58.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.837+0000 7fddf8226640 1 Processor -- start 2024-04-22T06:31:58.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.837+0000 7fddf8226640 1 -- start start 2024-04-22T06:31:58.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.838+0000 7fddf8226640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddf006d470 0x7fddf015a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:58.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.838+0000 7fddf8226640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fddf0154f00 0x7fddf01f0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:58.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.838+0000 7fddf8226640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddf01552e0 0x7fddf014c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:58.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.838+0000 7fddf8226640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fddf00c6bd0 con 0x7fddf01552e0 2024-04-22T06:31:58.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.838+0000 7fddf8226640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fddf00c63d0 con 0x7fddf006d470 2024-04-22T06:31:58.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.838+0000 7fddf8226640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fddf00c6d50 con 0x7fddf0154f00 2024-04-22T06:31:58.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.838+0000 7fddf67c3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddf01552e0 0x7fddf014c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:58.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.838+0000 7fddf67c3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddf01552e0 0x7fddf014c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55164/0 (socket says 172.21.15.44:55164) 2024-04-22T06:31:58.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.838+0000 7fddf67c3640 1 -- 172.21.15.44:0/308725484 learned_addr learned my addr 172.21.15.44:0/308725484 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:58.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.838+0000 7fddf5fc2640 1 --2- 172.21.15.44:0/308725484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddf006d470 0x7fddf015a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:58.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.838+0000 7fddf57c1640 1 --2- 172.21.15.44:0/308725484 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fddf0154f00 0x7fddf01f0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:58.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.838+0000 7fddf67c3640 1 -- 172.21.15.44:0/308725484 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fddf0154f00 msgr2=0x7fddf01f0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:58.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.838+0000 7fddf67c3640 1 --2- 172.21.15.44:0/308725484 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fddf0154f00 0x7fddf01f0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:58.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.838+0000 7fddf67c3640 1 -- 172.21.15.44:0/308725484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddf006d470 msgr2=0x7fddf015a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:58.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.838+0000 7fddf67c3640 1 --2- 172.21.15.44:0/308725484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddf006d470 0x7fddf015a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:58.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.838+0000 7fddf67c3640 1 -- 172.21.15.44:0/308725484 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf015ab90 con 0x7fddf01552e0 2024-04-22T06:31:58.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.839+0000 7fddf67c3640 1 --2- 172.21.15.44:0/308725484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddf01552e0 0x7fddf014c850 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdde800c7d0 tx=0x7fdde800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:58.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.839+0000 7fdde67fc640 1 -- 172.21.15.44:0/308725484 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde8003b70 con 0x7fddf01552e0 2024-04-22T06:31:58.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.839+0000 7fdde67fc640 1 -- 172.21.15.44:0/308725484 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdde8003d30 con 0x7fddf01552e0 2024-04-22T06:31:58.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.839+0000 7fdde67fc640 1 -- 172.21.15.44:0/308725484 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde800aa30 con 0x7fddf01552e0 2024-04-22T06:31:58.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.839+0000 7fddf8226640 1 -- 172.21.15.44:0/308725484 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddf01f13a0 con 0x7fddf01552e0 2024-04-22T06:31:58.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.839+0000 7fddf8226640 1 -- 172.21.15.44:0/308725484 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddf0159610 con 0x7fddf01552e0 2024-04-22T06:31:58.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.840+0000 7fdde67fc640 1 -- 172.21.15.44:0/308725484 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdde800abd0 con 0x7fddf01552e0 2024-04-22T06:31:58.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.841+0000 7fdde67fc640 1 --2- 172.21.15.44:0/308725484 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fddd406c920 0x7fddd406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:58.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.841+0000 7fdde67fc640 1 -- 172.21.15.44:0/308725484 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fdde808bc00 con 0x7fddf01552e0 2024-04-22T06:31:58.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.841+0000 7fddf5fc2640 1 --2- 172.21.15.44:0/308725484 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fddd406c920 0x7fddd406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:58.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.841+0000 7fddf5fc2640 1 --2- 172.21.15.44:0/308725484 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fddd406c920 0x7fddd406ede0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fddec007f70 tx=0x7fddec007f00 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:58.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.842+0000 7fddfae0f740 1 Processor -- start 2024-04-22T06:31:58.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.842+0000 7fddfae0f740 1 -- start start 2024-04-22T06:31:58.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.843+0000 7fddfae0f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631e5e04e40 0x5631e5e0df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:58.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.843+0000 7fddfae0f740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5631e5dc5ec0 0x5631e5e0e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:58.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.843+0000 7fddfae0f740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5631e5dc1ca0 0x5631e5de9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:58.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.843+0000 7fddfae0f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5631e5d28b10 con 0x5631e5e04e40 2024-04-22T06:31:58.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.843+0000 7fddfae0f740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5631e5cc6240 con 0x5631e5dc1ca0 2024-04-22T06:31:58.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.843+0000 7fddfae0f740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5631e5d24ad0 con 0x5631e5dc5ec0 2024-04-22T06:31:58.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.843+0000 7fddf5fc2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631e5e04e40 0x5631e5e0df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:58.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.843+0000 7fddf57c1640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5631e5dc5ec0 0x5631e5e0e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:58.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.843+0000 7fddf67c3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5631e5dc1ca0 0x5631e5de9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:58.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.843+0000 7fddf5fc2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631e5e04e40 0x5631e5e0df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55168/0 (socket says 172.21.15.44:55168) 2024-04-22T06:31:58.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.843+0000 7fddf5fc2640 1 -- 172.21.15.44:0/3027290377 learned_addr learned my addr 172.21.15.44:0/3027290377 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:58.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.843+0000 7fddf57c1640 1 --2- 172.21.15.44:0/3027290377 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5631e5dc5ec0 0x5631e5e0e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.44:45618/0 (socket says 172.21.15.44:45618) 2024-04-22T06:31:58.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.843+0000 7fddf67c3640 1 -- 172.21.15.44:0/3027290377 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5631e5dc5ec0 msgr2=0x5631e5e0e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:58.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.843+0000 7fddf67c3640 1 --2- 172.21.15.44:0/3027290377 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5631e5dc5ec0 0x5631e5e0e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:58.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.843+0000 7fddf67c3640 1 -- 172.21.15.44:0/3027290377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631e5e04e40 msgr2=0x5631e5e0df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:58.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.843+0000 7fddf67c3640 1 --2- 172.21.15.44:0/3027290377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631e5e04e40 0x5631e5e0df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:58.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.843+0000 7fddf67c3640 1 -- 172.21.15.44:0/3027290377 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631e5a3b2d0 con 0x5631e5dc1ca0 2024-04-22T06:31:58.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.843+0000 7fddf67c3640 1 --2- 172.21.15.44:0/3027290377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5631e5dc1ca0 0x5631e5de9ee0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdde8089d90 tx=0x7fdde8058fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=924f7070f182b549 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:58.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.843+0000 7fddd37fe640 1 -- 172.21.15.44:0/3027290377 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde8071020 con 0x5631e5dc1ca0 2024-04-22T06:31:58.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.843+0000 7fddd37fe640 1 -- 172.21.15.44:0/3027290377 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdde80621a0 con 0x5631e5dc1ca0 2024-04-22T06:31:58.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.843+0000 7fddd37fe640 1 -- 172.21.15.44:0/3027290377 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde8055ce0 con 0x5631e5dc1ca0 2024-04-22T06:31:58.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.844+0000 7fddfae0f740 1 -- 172.21.15.44:0/3027290377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5631e5dc1ca0 msgr2=0x5631e5de9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:31:58.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.844+0000 7fddfae0f740 1 --2- 172.21.15.44:0/3027290377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5631e5dc1ca0 0x5631e5de9ee0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdde8089d90 tx=0x7fdde8058fd0 comp rx=0 tx=0).stop 2024-04-22T06:31:58.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.844+0000 7fddfae0f740 1 -- 172.21.15.44:0/3027290377 shutdown_connections 2024-04-22T06:31:58.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.844+0000 7fddfae0f740 1 --2- 172.21.15.44:0/3027290377 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5631e5dc5ec0 0x5631e5e0e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:58.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.844+0000 7fddfae0f740 1 --2- 172.21.15.44:0/3027290377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5631e5dc1ca0 0x5631e5de9ee0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:58.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.844+0000 7fddfae0f740 1 --2- 172.21.15.44:0/3027290377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631e5e04e40 0x5631e5e0df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:58.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.844+0000 7fddfae0f740 1 -- 172.21.15.44:0/3027290377 >> 172.21.15.44:0/3027290377 conn(0x5631e5dc0990 msgr2=0x5631e5dc3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:31:58.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.844+0000 7fddfae0f740 1 -- 172.21.15.44:0/3027290377 shutdown_connections 2024-04-22T06:31:58.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.844+0000 7fddfae0f740 1 -- 172.21.15.44:0/3027290377 wait complete. 2024-04-22T06:31:58.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.845+0000 7fddfae0f740 1 -- start start 2024-04-22T06:31:58.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.845+0000 7fddfae0f740 1 Processor -- start 2024-04-22T06:31:58.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.846+0000 7fddfae0f740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5631e5dc5ec0 0x5631e5e15520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:58.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.846+0000 7fddfae0f740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5631e5e04e40 0x5631e5dbdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:58.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.846+0000 7fddfae0f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631e5e192e0 0x5631e5dbe290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:58.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.846+0000 7fddfae0f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5631e5cc6240 con 0x5631e5e192e0 2024-04-22T06:31:58.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.846+0000 7fddfae0f740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5631e5cb1190 con 0x5631e5e04e40 2024-04-22T06:31:58.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.846+0000 7fddfae0f740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5631e5dc4250 con 0x5631e5dc5ec0 2024-04-22T06:31:58.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.846+0000 7fddfae0f740 10 client.0 _ll_register_callbacks cb 0x5631e5cac8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T06:31:58.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.846+0000 7fddf67c3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631e5e192e0 0x5631e5dbe290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:58.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.846+0000 7fddf57c1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5631e5e04e40 0x5631e5dbdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:58.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.846+0000 7fddf5fc2640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5631e5dc5ec0 0x5631e5e15520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:58.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.846+0000 7fddf67c3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631e5e192e0 0x5631e5dbe290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55178/0 (socket says 172.21.15.44:55178) 2024-04-22T06:31:58.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.846+0000 7fddf57c1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5631e5e04e40 0x5631e5dbdd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:40474/0 (socket says 172.21.15.44:40474) 2024-04-22T06:31:58.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.846+0000 7fddf67c3640 1 -- 172.21.15.44:0/3535114634 learned_addr learned my addr 172.21.15.44:0/3535114634 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:31:58.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.846+0000 7fddf57c1640 1 -- 172.21.15.44:0/3535114634 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5631e5dc5ec0 msgr2=0x5631e5e15520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:58.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.846+0000 7fddf57c1640 1 --2- 172.21.15.44:0/3535114634 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5631e5dc5ec0 0x5631e5e15520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:58.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.846+0000 7fddf57c1640 1 -- 172.21.15.44:0/3535114634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631e5e192e0 msgr2=0x5631e5dbe290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:31:58.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.846+0000 7fddf57c1640 1 --2- 172.21.15.44:0/3535114634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631e5e192e0 0x5631e5dbe290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:31:58.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.846+0000 7fddf57c1640 1 -- 172.21.15.44:0/3535114634 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631e5b056b0 con 0x5631e5e04e40 2024-04-22T06:31:58.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.846+0000 7fddf5fc2640 1 --2- 172.21.15.44:0/3535114634 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5631e5dc5ec0 0x5631e5e15520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:31:58.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.846+0000 7fddf67c3640 1 --2- 172.21.15.44:0/3535114634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631e5e192e0 0x5631e5dbe290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:31:58.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.846+0000 7fddf57c1640 1 --2- 172.21.15.44:0/3535114634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5631e5e04e40 0x5631e5dbdd50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fddf006c770 tx=0x7fdde0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49c981967190cb4a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:31:58.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.847+0000 7fddd0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2024-04-22T06:31:58.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.847+0000 7fddd0ff9640 1 -- 172.21.15.44:0/3535114634 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde0035040 con 0x5631e5e04e40 2024-04-22T06:31:58.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.847+0000 7fddd0ff9640 1 -- 172.21.15.44:0/3535114634 <== mon.1 v2:172.21.15.70:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fdde003a420 con 0x5631e5e04e40 2024-04-22T06:31:58.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.847+0000 7fddfae0f740 1 -- 172.21.15.44:0/3535114634 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdde80621a0 con 0x5631e5e04e40 2024-04-22T06:31:58.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.847+0000 7fddd0ff9640 1 -- 172.21.15.44:0/3535114634 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fdde003a620 con 0x5631e5e04e40 2024-04-22T06:31:58.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.847+0000 7fddfae0f740 10 client.4840 fetch_fsmap learned FSMap version 18 2024-04-22T06:31:58.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.847+0000 7fddfae0f740 1 -- 172.21.15.44:0/3535114634 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x5631e5e0cc70 con 0x5631e5e04e40 2024-04-22T06:31:58.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.847+0000 7fddd0ff9640 1 -- 172.21.15.44:0/3535114634 <== mon.1 v2:172.21.15.70:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdde80621a0 con 0x5631e5e04e40 2024-04-22T06:31:58.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.847+0000 7fddd0ff9640 1 -- 172.21.15.44:0/3535114634 <== mon.1 v2:172.21.15.70:3300/0 5 ==== fsmap(e 18) ==== 4792+0+0 (secure 0 0 0) 0x7fdde004d050 con 0x5631e5e04e40 2024-04-22T06:31:58.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.847+0000 7fddfae0f740 10 client.4840 fetch_fsmap finished waiting for FSMap version 18 2024-04-22T06:31:58.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.847+0000 7fddfae0f740 10 client.4840 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-22T06:31:58.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.847+0000 7fddfae0f740 20 client.4840 populate_metadata read hostname 'smithi044' 2024-04-22T06:31:58.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.847+0000 7fddfae0f740 1 --2- 172.21.15.44:0/3535114634 >> [v2:172.21.15.44:6832/1497302315,v1:172.21.15.44:6834/1497302315] conn(0x5631e5e0c080 0x5631e5e24560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:31:58.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.847+0000 7fddfae0f740 4 client.4840 mds_command: new command op to 4287 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T06:31:58.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.847+0000 7fddfae0f740 1 -- 172.21.15.44:0/3535114634 --> [v2:172.21.15.44:6832/1497302315,v1:172.21.15.44:6834/1497302315] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631e5d36530 con 0x5631e5e0c080 2024-04-22T06:31:58.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.847+0000 7fddf5fc2640 1 --2- 172.21.15.44:0/3535114634 >> [v2:172.21.15.44:6832/1497302315,v1:172.21.15.44:6834/1497302315] conn(0x5631e5e0c080 0x5631e5e24560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:31:58.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.848+0000 7fddf5fc2640 1 --2- 172.21.15.44:0/3535114634 >> [v2:172.21.15.44:6832/1497302315,v1:172.21.15.44:6834/1497302315] conn(0x5631e5e0c080 0x5631e5e24560 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f470354193b3b64d server_cookie=6d91a2123209c7d6 in_seq=0 out_seq=0 2024-04-22T06:31:58.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:31:58.848+0000 7fddd0ff9640 10 client.4840 ms_handle_connect on v2:172.21.15.44:6832/1497302315 2024-04-22T06:32:01.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.174+0000 7fddd0ff9640 1 -- 172.21.15.44:0/3535114634 <== mds.4 v2:172.21.15.44:6832/1497302315 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5631e5d36530 con 0x5631e5e0c080 2024-04-22T06:32:01.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.174+0000 7fddd0ff9640 10 client.4840 handle_command_reply: tid=0 2024-04-22T06:32:01.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.174+0000 7fddfae0f740 1 client.4840 shutdown 2024-04-22T06:32:01.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.174+0000 7fddfae0f740 1 -- 172.21.15.44:0/3535114634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5631e5e04e40 msgr2=0x5631e5dbdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:01.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.174+0000 7fddfae0f740 1 --2- 172.21.15.44:0/3535114634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5631e5e04e40 0x5631e5dbdd50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fddf006c770 tx=0x7fdde0004730 comp rx=0 tx=0).stop 2024-04-22T06:32:01.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.174+0000 7fddfae0f740 1 -- 172.21.15.44:0/3535114634 shutdown_connections 2024-04-22T06:32:01.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.174+0000 7fddfae0f740 1 --2- 172.21.15.44:0/3535114634 >> [v2:172.21.15.44:6832/1497302315,v1:172.21.15.44:6834/1497302315] conn(0x5631e5e0c080 0x5631e5e24560 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.174+0000 7fddfae0f740 1 --2- 172.21.15.44:0/3535114634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631e5e192e0 0x5631e5dbe290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.174+0000 7fddfae0f740 1 --2- 172.21.15.44:0/3535114634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5631e5e04e40 0x5631e5dbdd50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.174+0000 7fddfae0f740 1 --2- 172.21.15.44:0/3535114634 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5631e5dc5ec0 0x5631e5e15520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.174+0000 7fddd0ff9640 0 client.4840 ms_handle_reset on v2:172.21.15.44:6832/1497302315 2024-04-22T06:32:01.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.174+0000 7fddfae0f740 1 -- 172.21.15.44:0/3535114634 >> 172.21.15.44:0/3535114634 conn(0x5631e5dc1ca0 msgr2=0x5631e5de5d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:01.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.174+0000 7fddfae0f740 1 -- 172.21.15.44:0/3535114634 shutdown_connections 2024-04-22T06:32:01.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.174+0000 7fddfae0f740 1 -- 172.21.15.44:0/3535114634 wait complete. 2024-04-22T06:32:01.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.175+0000 7fddfae0f740 20 client.4840 trim_cache size 0 max 16384 2024-04-22T06:32:01.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.195+0000 7fddfae0f740 1 Processor -- start 2024-04-22T06:32:01.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.195+0000 7fddfae0f740 1 -- start start 2024-04-22T06:32:01.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.196+0000 7fddfae0f740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5631e5dc5ec0 0x5631e5e02070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.196+0000 7fddfae0f740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5631e5e04e40 0x5631e5e0a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.196+0000 7fddfae0f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631e5e192e0 0x5631e5e0aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.196+0000 7fddfae0f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fddf01f4fd0 con 0x5631e5e192e0 2024-04-22T06:32:01.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.196+0000 7fddfae0f740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5631e5e0cf50 con 0x5631e5e04e40 2024-04-22T06:32:01.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.196+0000 7fddfae0f740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5631e5e0bf00 con 0x5631e5dc5ec0 2024-04-22T06:32:01.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.196+0000 7fddf67c3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631e5e192e0 0x5631e5e0aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.196+0000 7fddf67c3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631e5e192e0 0x5631e5e0aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55180/0 (socket says 172.21.15.44:55180) 2024-04-22T06:32:01.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.196+0000 7fddf67c3640 1 -- 172.21.15.44:0/2640060152 learned_addr learned my addr 172.21.15.44:0/2640060152 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:01.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.196+0000 7fddf5fc2640 1 --2- 172.21.15.44:0/2640060152 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5631e5dc5ec0 0x5631e5e02070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.196+0000 7fddf57c1640 1 --2- 172.21.15.44:0/2640060152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5631e5e04e40 0x5631e5e0a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.196+0000 7fddf67c3640 1 -- 172.21.15.44:0/2640060152 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5631e5dc5ec0 msgr2=0x5631e5e02070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:01.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.196+0000 7fddf67c3640 1 --2- 172.21.15.44:0/2640060152 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5631e5dc5ec0 0x5631e5e02070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.196+0000 7fddf67c3640 1 -- 172.21.15.44:0/2640060152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5631e5e04e40 msgr2=0x5631e5e0a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:01.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.196+0000 7fddf67c3640 1 --2- 172.21.15.44:0/2640060152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5631e5e04e40 0x5631e5e0a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.196+0000 7fddf67c3640 1 -- 172.21.15.44:0/2640060152 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631e5dc0e60 con 0x5631e5e192e0 2024-04-22T06:32:01.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.197+0000 7fddf67c3640 1 --2- 172.21.15.44:0/2640060152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631e5e192e0 0x5631e5e0aa20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdde8059830 tx=0x7fdde805b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6568aa5edf714b14 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:01.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.197+0000 7fddd2ffd640 1 -- 172.21.15.44:0/2640060152 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde8071040 con 0x5631e5e192e0 2024-04-22T06:32:01.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.197+0000 7fddd2ffd640 1 -- 172.21.15.44:0/2640060152 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdde8076070 con 0x5631e5e192e0 2024-04-22T06:32:01.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.197+0000 7fddd2ffd640 1 -- 172.21.15.44:0/2640060152 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde8055910 con 0x5631e5e192e0 2024-04-22T06:32:01.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.197+0000 7fddfae0f740 1 -- 172.21.15.44:0/2640060152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631e5e192e0 msgr2=0x5631e5e0aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:01.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.198+0000 7fddfae0f740 1 --2- 172.21.15.44:0/2640060152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631e5e192e0 0x5631e5e0aa20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdde8059830 tx=0x7fdde805b000 comp rx=0 tx=0).stop 2024-04-22T06:32:01.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.198+0000 7fddfae0f740 1 -- 172.21.15.44:0/2640060152 shutdown_connections 2024-04-22T06:32:01.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.198+0000 7fddfae0f740 1 --2- 172.21.15.44:0/2640060152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631e5e192e0 0x5631e5e0aa20 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.198+0000 7fddfae0f740 1 --2- 172.21.15.44:0/2640060152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5631e5e04e40 0x5631e5e0a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.198+0000 7fddfae0f740 1 --2- 172.21.15.44:0/2640060152 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5631e5dc5ec0 0x5631e5e02070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.198+0000 7fddfae0f740 1 -- 172.21.15.44:0/2640060152 >> 172.21.15.44:0/2640060152 conn(0x5631e5dc1ca0 msgr2=0x5631e5dbdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:01.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.198+0000 7fddfae0f740 1 -- 172.21.15.44:0/2640060152 shutdown_connections 2024-04-22T06:32:01.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.198+0000 7fddfae0f740 1 -- 172.21.15.44:0/2640060152 wait complete. 2024-04-22T06:32:01.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.198+0000 7fddfae0f740 1 -- start start 2024-04-22T06:32:01.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.198+0000 7fddfae0f740 1 Processor -- start 2024-04-22T06:32:01.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.199+0000 7fddfae0f740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5631e5dc5ec0 0x5631e5e08080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.199+0000 7fddfae0f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631e5e04e40 0x5631e5dbe150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.199+0000 7fddfae0f740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5631e5e192e0 0x5631e5e82760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.199+0000 7fddfae0f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5631e5ddb640 con 0x5631e5e04e40 2024-04-22T06:32:01.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.199+0000 7fddfae0f740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5631e5e0c4f0 con 0x5631e5dc5ec0 2024-04-22T06:32:01.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.199+0000 7fddfae0f740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5631e5ddbcf0 con 0x5631e5e192e0 2024-04-22T06:32:01.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.199+0000 7fddfae0f740 10 client.0 _ll_register_callbacks cb 0x7fdde00094b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T06:32:01.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.200+0000 7fddf57c1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631e5e04e40 0x5631e5dbe150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.200+0000 7fddf57c1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631e5e04e40 0x5631e5dbe150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55188/0 (socket says 172.21.15.44:55188) 2024-04-22T06:32:01.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.200+0000 7fddf57c1640 1 -- 172.21.15.44:0/1452865556 learned_addr learned my addr 172.21.15.44:0/1452865556 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:01.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.200+0000 7fddf67c3640 1 --2- 172.21.15.44:0/1452865556 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5631e5e192e0 0x5631e5e82760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.200+0000 7fddf5fc2640 1 --2- 172.21.15.44:0/1452865556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5631e5dc5ec0 0x5631e5e08080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.200+0000 7fddf57c1640 1 -- 172.21.15.44:0/1452865556 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5631e5e192e0 msgr2=0x5631e5e82760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:01.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.200+0000 7fddf57c1640 1 --2- 172.21.15.44:0/1452865556 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5631e5e192e0 0x5631e5e82760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.200+0000 7fddf57c1640 1 -- 172.21.15.44:0/1452865556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5631e5dc5ec0 msgr2=0x5631e5e08080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:01.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.200+0000 7fddf57c1640 1 --2- 172.21.15.44:0/1452865556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5631e5dc5ec0 0x5631e5e08080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.200+0000 7fddf57c1640 1 -- 172.21.15.44:0/1452865556 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631e5e02e10 con 0x5631e5e04e40 2024-04-22T06:32:01.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.200+0000 7fddf57c1640 1 --2- 172.21.15.44:0/1452865556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631e5e04e40 0x5631e5dbe150 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdde0001eb0 tx=0x7fdde0031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=247ea8c0dcf7f1e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:01.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.200+0000 7fddd0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T06:32:01.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.200+0000 7fddd0ff9640 1 -- 172.21.15.44:0/1452865556 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde0035040 con 0x5631e5e04e40 2024-04-22T06:32:01.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.200+0000 7fddd0ff9640 1 -- 172.21.15.44:0/1452865556 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fdde0003610 con 0x5631e5e04e40 2024-04-22T06:32:01.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.200+0000 7fddfae0f740 1 -- 172.21.15.44:0/1452865556 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdde8076070 con 0x5631e5e04e40 2024-04-22T06:32:01.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.201+0000 7fddd0ff9640 1 -- 172.21.15.44:0/1452865556 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fdde005d050 con 0x5631e5e04e40 2024-04-22T06:32:01.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.201+0000 7fddfae0f740 10 client.4929 fetch_fsmap learned FSMap version 18 2024-04-22T06:32:01.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.201+0000 7fddfae0f740 1 -- 172.21.15.44:0/1452865556 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x5631e5ddb8a0 con 0x5631e5e04e40 2024-04-22T06:32:01.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.201+0000 7fddd0ff9640 1 -- 172.21.15.44:0/1452865556 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdde8076070 con 0x5631e5e04e40 2024-04-22T06:32:01.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.201+0000 7fddd0ff9640 1 -- 172.21.15.44:0/1452865556 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 18) ==== 4792+0+0 (secure 0 0 0) 0x7fdde0060020 con 0x5631e5e04e40 2024-04-22T06:32:01.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.201+0000 7fddfae0f740 10 client.4929 fetch_fsmap finished waiting for FSMap version 18 2024-04-22T06:32:01.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.201+0000 7fddfae0f740 10 client.4929 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-22T06:32:01.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.201+0000 7fddfae0f740 20 client.4929 populate_metadata read hostname 'smithi044' 2024-04-22T06:32:01.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.201+0000 7fddfae0f740 1 --2- 172.21.15.44:0/1452865556 >> [v2:172.21.15.44:6832/1497302315,v1:172.21.15.44:6834/1497302315] conn(0x5631e5e7fa40 0x5631e5e7fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.201+0000 7fddfae0f740 4 client.4929 mds_command: new command op to 4287 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T06:32:01.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.201+0000 7fddfae0f740 1 -- 172.21.15.44:0/1452865556 --> [v2:172.21.15.44:6832/1497302315,v1:172.21.15.44:6834/1497302315] -- command(tid 0: {"prefix": "status"}) -- 0x5631e5dc45c0 con 0x5631e5e7fa40 2024-04-22T06:32:01.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.201+0000 7fddf5fc2640 1 --2- 172.21.15.44:0/1452865556 >> [v2:172.21.15.44:6832/1497302315,v1:172.21.15.44:6834/1497302315] conn(0x5631e5e7fa40 0x5631e5e7fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.202+0000 7fddf5fc2640 1 --2- 172.21.15.44:0/1452865556 >> [v2:172.21.15.44:6832/1497302315,v1:172.21.15.44:6834/1497302315] conn(0x5631e5e7fa40 0x5631e5e7fe20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a596c5a304ba4129 server_cookie=6cb522029cc4d3f9 in_seq=0 out_seq=0 2024-04-22T06:32:01.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.202+0000 7fddd0ff9640 10 client.4929 ms_handle_connect on v2:172.21.15.44:6832/1497302315 2024-04-22T06:32:01.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.516+0000 7fddd0ff9640 1 -- 172.21.15.44:0/1452865556 <== mds.4 v2:172.21.15.44:6832/1497302315 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5631e5dc45c0 con 0x5631e5e7fa40 2024-04-22T06:32:01.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.516+0000 7fddd0ff9640 10 client.4929 handle_command_reply: tid=0 2024-04-22T06:32:01.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.516+0000 7fddfae0f740 1 client.4929 shutdown 2024-04-22T06:32:01.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.516+0000 7fddfae0f740 1 -- 172.21.15.44:0/1452865556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631e5e04e40 msgr2=0x5631e5dbe150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:01.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.516+0000 7fddfae0f740 1 --2- 172.21.15.44:0/1452865556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631e5e04e40 0x5631e5dbe150 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdde0001eb0 tx=0x7fdde0031000 comp rx=0 tx=0).stop 2024-04-22T06:32:01.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.516+0000 7fddfae0f740 1 -- 172.21.15.44:0/1452865556 shutdown_connections 2024-04-22T06:32:01.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.516+0000 7fddfae0f740 1 --2- 172.21.15.44:0/1452865556 >> [v2:172.21.15.44:6832/1497302315,v1:172.21.15.44:6834/1497302315] conn(0x5631e5e7fa40 0x5631e5e7fe20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.516+0000 7fddfae0f740 1 --2- 172.21.15.44:0/1452865556 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5631e5e192e0 0x5631e5e82760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.516+0000 7fddfae0f740 1 --2- 172.21.15.44:0/1452865556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5631e5e04e40 0x5631e5dbe150 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.516+0000 7fddfae0f740 1 --2- 172.21.15.44:0/1452865556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5631e5dc5ec0 0x5631e5e08080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.516+0000 7fddd0ff9640 0 client.4929 ms_handle_reset on v2:172.21.15.44:6832/1497302315 2024-04-22T06:32:01.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.516+0000 7fddfae0f740 1 -- 172.21.15.44:0/1452865556 >> 172.21.15.44:0/1452865556 conn(0x5631e5dc1ca0 msgr2=0x5631e5de18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:01.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.516+0000 7fddfae0f740 1 -- 172.21.15.44:0/1452865556 shutdown_connections 2024-04-22T06:32:01.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.516+0000 7fddfae0f740 1 -- 172.21.15.44:0/1452865556 wait complete. 2024-04-22T06:32:01.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.516+0000 7fddfae0f740 20 client.4929 trim_cache size 0 max 16384 2024-04-22T06:32:01.517 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-22T06:32:01.517 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "9b0a34ca-403d-4513-8a66-d3749ba559aa", 2024-04-22T06:32:01.517 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 4, 2024-04-22T06:32:01.517 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 4287, 2024-04-22T06:32:01.517 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-04-22T06:32:01.517 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-04-22T06:32:01.517 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-04-22T06:32:01.517 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 11.364006064, 2024-04-22T06:32:01.517 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 18, 2024-04-22T06:32:01.517 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 27, 2024-04-22T06:32:01.517 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 27, 2024-04-22T06:32:01.517 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 61.424764013999997 2024-04-22T06:32:01.518 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-22T06:32:01.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.517+0000 7fddf8226640 1 -- 172.21.15.44:0/308725484 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fddd406c920 msgr2=0x7fddd406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:01.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.517+0000 7fddf8226640 1 --2- 172.21.15.44:0/308725484 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fddd406c920 0x7fddd406ede0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fddec007f70 tx=0x7fddec007f00 comp rx=0 tx=0).stop 2024-04-22T06:32:01.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.517+0000 7fddf8226640 1 -- 172.21.15.44:0/308725484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddf01552e0 msgr2=0x7fddf014c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:01.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.517+0000 7fddf8226640 1 --2- 172.21.15.44:0/308725484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddf01552e0 0x7fddf014c850 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdde800c7d0 tx=0x7fdde800cc90 comp rx=0 tx=0).stop 2024-04-22T06:32:01.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.517+0000 7fddf8226640 1 -- 172.21.15.44:0/308725484 shutdown_connections 2024-04-22T06:32:01.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.517+0000 7fddf8226640 1 --2- 172.21.15.44:0/308725484 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fddd406c920 0x7fddd406ede0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.517+0000 7fddf8226640 1 --2- 172.21.15.44:0/308725484 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fddf0154f00 0x7fddf01f0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.517+0000 7fddf8226640 1 --2- 172.21.15.44:0/308725484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddf01552e0 0x7fddf014c850 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.517+0000 7fddf8226640 1 --2- 172.21.15.44:0/308725484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddf006d470 0x7fddf015a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.517+0000 7fddf8226640 1 -- 172.21.15.44:0/308725484 >> 172.21.15.44:0/308725484 conn(0x7fddf00733c0 msgr2=0x7fddf00d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:01.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.517+0000 7fddf8226640 1 -- 172.21.15.44:0/308725484 shutdown_connections 2024-04-22T06:32:01.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.517+0000 7fddf8226640 1 -- 172.21.15.44:0/308725484 wait complete. 2024-04-22T06:32:01.530 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-22T06:32:01.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.641+0000 7f39e04f2640 1 Processor -- start 2024-04-22T06:32:01.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.641+0000 7f39e04f2640 1 -- start start 2024-04-22T06:32:01.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.642+0000 7f39e04f2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f39d81552e0 0x7f39d80cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.642+0000 7f39e04f2640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39d815ccf0 con 0x7f39d8154f00 2024-04-22T06:32:01.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.642+0000 7f39e04f2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39d80c5b30 con 0x7f39d806d470 2024-04-22T06:32:01.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.642+0000 7f39e04f2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f39d80c6740 con 0x7f39d81552e0 2024-04-22T06:32:01.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.642+0000 7f39de28e640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f39d8154f00 0x7f39d815c5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:38708/0 (socket says 172.21.15.44:38708) 2024-04-22T06:32:01.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.642+0000 7f39de28e640 1 -- 172.21.15.44:0/3824089814 learned_addr learned my addr 172.21.15.44:0/3824089814 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:32:01.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.642+0000 7f39dda8d640 1 --2- 172.21.15.44:0/3824089814 >> v2:172.21.15.70:3300/0 conn(0x7f39d81552e0 0x7f39d80cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.642+0000 7f39dd28c640 1 -- 172.21.15.44:0/3824089814 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1556965727 0 0) 0x7f39d815ccf0 con 0x7f39d8154f00 2024-04-22T06:32:01.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.642+0000 7f39dd28c640 1 -- 172.21.15.44:0/3824089814 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39c4003520 con 0x7f39d8154f00 2024-04-22T06:32:01.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.642+0000 7f39dd28c640 1 -- 172.21.15.44:0/3824089814 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 587799724 0 0) 0x7f39d80c5b30 con 0x7f39d806d470 2024-04-22T06:32:01.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.642+0000 7f39dd28c640 1 -- 172.21.15.44:0/3824089814 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39d815ccf0 con 0x7f39d806d470 2024-04-22T06:32:01.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.642+0000 7f39dda8d640 1 -- 172.21.15.44:0/3824089814 >> v1:172.21.15.70:6789/0 conn(0x7f39d806d470 legacy=0x7f39d80d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:01.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.642+0000 7f39dda8d640 1 -- 172.21.15.44:0/3824089814 >> v1:172.21.15.44:6789/0 conn(0x7f39d8154f00 legacy=0x7f39d815c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:01.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.642+0000 7f39dda8d640 1 -- 172.21.15.44:0/3824089814 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d80c5e10 con 0x7f39d81552e0 2024-04-22T06:32:01.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.643+0000 7f39dda8d640 1 --2- 172.21.15.44:0/3824089814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39d81552e0 0x7f39d80cb430 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f39c80097a0 tx=0x7f39c802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2936b210e72a57f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:01.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.643+0000 7f39dd28c640 1 -- 172.21.15.44:0/3824089814 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c8035020 con 0x7f39d81552e0 2024-04-22T06:32:01.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.643+0000 7f39dd28c640 1 -- 172.21.15.44:0/3824089814 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39c8031830 con 0x7f39d81552e0 2024-04-22T06:32:01.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.643+0000 7f39dd28c640 1 -- 172.21.15.44:0/3824089814 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c8031c00 con 0x7f39d81552e0 2024-04-22T06:32:01.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.644+0000 7f39e04f2640 1 -- 172.21.15.44:0/3824089814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39d81552e0 msgr2=0x7f39d80cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:01.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.644+0000 7f39e04f2640 1 --2- 172.21.15.44:0/3824089814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39d81552e0 0x7f39d80cb430 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f39c80097a0 tx=0x7f39c802b8d0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.644+0000 7f39e04f2640 1 -- 172.21.15.44:0/3824089814 shutdown_connections 2024-04-22T06:32:01.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.644+0000 7f39e04f2640 1 --2- 172.21.15.44:0/3824089814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39d81552e0 0x7f39d80cb430 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.644+0000 7f39e04f2640 1 -- 172.21.15.44:0/3824089814 >> 172.21.15.44:0/3824089814 conn(0x7f39d80733c0 msgr2=0x7f39d80737d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:01.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.644+0000 7f39e04f2640 1 -- 172.21.15.44:0/3824089814 shutdown_connections 2024-04-22T06:32:01.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.644+0000 7f39e04f2640 1 -- 172.21.15.44:0/3824089814 wait complete. 2024-04-22T06:32:01.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.645+0000 7f39e04f2640 1 Processor -- start 2024-04-22T06:32:01.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.645+0000 7f39e04f2640 1 -- start start 2024-04-22T06:32:01.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.646+0000 7f39e04f2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39d806d470 0x7f39d80cec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.646+0000 7f39e04f2640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39d8154f00 0x7f39d80cd530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.646+0000 7f39e04f2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39d81552e0 0x7f39d814a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.646+0000 7f39e04f2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f39d80c6bd0 con 0x7f39d81552e0 2024-04-22T06:32:01.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.646+0000 7f39e04f2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f39d80c63d0 con 0x7f39d806d470 2024-04-22T06:32:01.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.646+0000 7f39e04f2640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f39d80c6d50 con 0x7f39d8154f00 2024-04-22T06:32:01.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.646+0000 7f39dea8f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39d81552e0 0x7f39d814a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.646+0000 7f39dea8f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39d81552e0 0x7f39d814a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55198/0 (socket says 172.21.15.44:55198) 2024-04-22T06:32:01.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.646+0000 7f39dda8d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39d8154f00 0x7f39d80cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.646+0000 7f39dea8f640 1 -- 172.21.15.44:0/927154971 learned_addr learned my addr 172.21.15.44:0/927154971 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:01.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.646+0000 7f39de28e640 1 --2- 172.21.15.44:0/927154971 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39d806d470 0x7f39d80cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.646+0000 7f39dea8f640 1 -- 172.21.15.44:0/927154971 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39d8154f00 msgr2=0x7f39d80cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:01.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.646+0000 7f39dea8f640 1 --2- 172.21.15.44:0/927154971 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39d8154f00 0x7f39d80cd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.646+0000 7f39dea8f640 1 -- 172.21.15.44:0/927154971 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39d806d470 msgr2=0x7f39d80cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:01.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.646+0000 7f39dea8f640 1 --2- 172.21.15.44:0/927154971 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39d806d470 0x7f39d80cec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.646+0000 7f39dea8f640 1 -- 172.21.15.44:0/927154971 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d81f4540 con 0x7f39d81552e0 2024-04-22T06:32:01.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.646+0000 7f39dda8d640 1 --2- 172.21.15.44:0/927154971 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39d8154f00 0x7f39d80cd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:32:01.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.646+0000 7f39de28e640 1 --2- 172.21.15.44:0/927154971 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39d806d470 0x7f39d80cec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:32:01.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.647+0000 7f39dea8f640 1 --2- 172.21.15.44:0/927154971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39d81552e0 0x7f39d814a250 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f39d000c7c0 tx=0x7f39d000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:01.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.647+0000 7f39ceffd640 1 -- 172.21.15.44:0/927154971 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d0003b70 con 0x7f39d81552e0 2024-04-22T06:32:01.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.647+0000 7f39ceffd640 1 -- 172.21.15.44:0/927154971 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39d0003d10 con 0x7f39d81552e0 2024-04-22T06:32:01.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.647+0000 7f39e04f2640 1 -- 172.21.15.44:0/927154971 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39d80b5340 con 0x7f39d81552e0 2024-04-22T06:32:01.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.647+0000 7f39ceffd640 1 -- 172.21.15.44:0/927154971 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d000a9f0 con 0x7f39d81552e0 2024-04-22T06:32:01.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.647+0000 7f39e04f2640 1 -- 172.21.15.44:0/927154971 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39d80b58c0 con 0x7f39d81552e0 2024-04-22T06:32:01.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.648+0000 7f39ceffd640 1 -- 172.21.15.44:0/927154971 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f39d000ab90 con 0x7f39d81552e0 2024-04-22T06:32:01.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.649+0000 7f39ceffd640 1 --2- 172.21.15.44:0/927154971 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f39bc06c8f0 0x7f39bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.649+0000 7f39ceffd640 1 -- 172.21.15.44:0/927154971 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f39d008be00 con 0x7f39d81552e0 2024-04-22T06:32:01.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.649+0000 7f39de28e640 1 --2- 172.21.15.44:0/927154971 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f39bc06c8f0 0x7f39bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.649+0000 7f39de28e640 1 --2- 172.21.15.44:0/927154971 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f39bc06c8f0 0x7f39bc06edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f39d40011e0 tx=0x7f39d4007d30 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:01.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.650+0000 7f39e31c0740 1 Processor -- start 2024-04-22T06:32:01.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.650+0000 7f39e31c0740 1 -- start start 2024-04-22T06:32:01.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.651+0000 7f39e31c0740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fe14eaae40 0x55fe14eb3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.651+0000 7f39e31c0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe14e6bec0 0x55fe14eb44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.651+0000 7f39e31c0740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fe14e67ca0 0x55fe14e8fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.651+0000 7f39e31c0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55fe14dceb10 con 0x55fe14e6bec0 2024-04-22T06:32:01.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.651+0000 7f39e31c0740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55fe14d6c240 con 0x55fe14eaae40 2024-04-22T06:32:01.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.651+0000 7f39e31c0740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55fe14dcaad0 con 0x55fe14e67ca0 2024-04-22T06:32:01.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.651+0000 7f39dda8d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe14e6bec0 0x55fe14eb44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.651+0000 7f39dda8d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe14e6bec0 0x55fe14eb44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55208/0 (socket says 172.21.15.44:55208) 2024-04-22T06:32:01.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.651+0000 7f39de28e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fe14eaae40 0x55fe14eb3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.651+0000 7f39dda8d640 1 -- 172.21.15.44:0/523843286 learned_addr learned my addr 172.21.15.44:0/523843286 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:01.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.651+0000 7f39dea8f640 1 --2- 172.21.15.44:0/523843286 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fe14e67ca0 0x55fe14e8fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.651+0000 7f39dea8f640 1 -- 172.21.15.44:0/523843286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fe14eaae40 msgr2=0x55fe14eb3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:01.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.651+0000 7f39dea8f640 1 --2- 172.21.15.44:0/523843286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fe14eaae40 0x55fe14eb3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.651+0000 7f39dea8f640 1 -- 172.21.15.44:0/523843286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe14e6bec0 msgr2=0x55fe14eb44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:01.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.651+0000 7f39dea8f640 1 --2- 172.21.15.44:0/523843286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe14e6bec0 0x55fe14eb44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.651+0000 7f39dea8f640 1 -- 172.21.15.44:0/523843286 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe14ae12d0 con 0x55fe14e67ca0 2024-04-22T06:32:01.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.651+0000 7f39dea8f640 1 --2- 172.21.15.44:0/523843286 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fe14e67ca0 0x55fe14e8fee0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f39d0089e60 tx=0x7f39d0059160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b98a38a15d08a17 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:01.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.652+0000 7f39bbfff640 1 -- 172.21.15.44:0/523843286 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d0057a40 con 0x55fe14e67ca0 2024-04-22T06:32:01.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.652+0000 7f39bbfff640 1 -- 172.21.15.44:0/523843286 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39d0055db0 con 0x55fe14e67ca0 2024-04-22T06:32:01.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.652+0000 7f39bbfff640 1 -- 172.21.15.44:0/523843286 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d0062000 con 0x55fe14e67ca0 2024-04-22T06:32:01.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.652+0000 7f39e31c0740 1 -- 172.21.15.44:0/523843286 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fe14e67ca0 msgr2=0x55fe14e8fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:01.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.652+0000 7f39e31c0740 1 --2- 172.21.15.44:0/523843286 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fe14e67ca0 0x55fe14e8fee0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f39d0089e60 tx=0x7f39d0059160 comp rx=0 tx=0).stop 2024-04-22T06:32:01.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.652+0000 7f39e31c0740 1 -- 172.21.15.44:0/523843286 shutdown_connections 2024-04-22T06:32:01.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.652+0000 7f39e31c0740 1 --2- 172.21.15.44:0/523843286 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fe14e67ca0 0x55fe14e8fee0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.652+0000 7f39e31c0740 1 --2- 172.21.15.44:0/523843286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe14e6bec0 0x55fe14eb44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.652+0000 7f39e31c0740 1 --2- 172.21.15.44:0/523843286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fe14eaae40 0x55fe14eb3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.652+0000 7f39e31c0740 1 -- 172.21.15.44:0/523843286 >> 172.21.15.44:0/523843286 conn(0x55fe14e66990 msgr2=0x55fe14e69290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:01.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.652+0000 7f39e31c0740 1 -- 172.21.15.44:0/523843286 shutdown_connections 2024-04-22T06:32:01.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.653+0000 7f39e31c0740 1 -- 172.21.15.44:0/523843286 wait complete. 2024-04-22T06:32:01.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.653+0000 7f39e31c0740 1 -- start start 2024-04-22T06:32:01.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.653+0000 7f39e31c0740 1 Processor -- start 2024-04-22T06:32:01.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.654+0000 7f39e31c0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe14e6bec0 0x55fe14e68660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.654+0000 7f39e31c0740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fe14eaae40 0x55fe14ebf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.654+0000 7f39e31c0740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fe14ea3890 0x55fe14e877c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.654+0000 7f39e31c0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55fe14e6a250 con 0x55fe14e6bec0 2024-04-22T06:32:01.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.654+0000 7f39e31c0740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55fe14d57190 con 0x55fe14eaae40 2024-04-22T06:32:01.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.654+0000 7f39e31c0740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55fe14e6a530 con 0x55fe14ea3890 2024-04-22T06:32:01.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.654+0000 7f39e31c0740 10 client.0 _ll_register_callbacks cb 0x55fe14d528b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T06:32:01.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.654+0000 7f39de28e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe14e6bec0 0x55fe14e68660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.654+0000 7f39de28e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe14e6bec0 0x55fe14e68660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55214/0 (socket says 172.21.15.44:55214) 2024-04-22T06:32:01.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.654+0000 7f39de28e640 1 -- 172.21.15.44:0/3725508931 learned_addr learned my addr 172.21.15.44:0/3725508931 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:01.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.654+0000 7f39dda8d640 1 --2- 172.21.15.44:0/3725508931 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fe14eaae40 0x55fe14ebf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.654+0000 7f39dea8f640 1 --2- 172.21.15.44:0/3725508931 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fe14ea3890 0x55fe14e877c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.654+0000 7f39de28e640 1 -- 172.21.15.44:0/3725508931 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fe14ea3890 msgr2=0x55fe14e877c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:01.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.654+0000 7f39de28e640 1 --2- 172.21.15.44:0/3725508931 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fe14ea3890 0x55fe14e877c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.654+0000 7f39de28e640 1 -- 172.21.15.44:0/3725508931 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fe14eaae40 msgr2=0x55fe14ebf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:01.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.654+0000 7f39de28e640 1 --2- 172.21.15.44:0/3725508931 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fe14eaae40 0x55fe14ebf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.654+0000 7f39de28e640 1 -- 172.21.15.44:0/3725508931 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe14bab6b0 con 0x55fe14e6bec0 2024-04-22T06:32:01.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.655+0000 7f39de28e640 1 --2- 172.21.15.44:0/3725508931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe14e6bec0 0x55fe14e68660 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f39d4014980 tx=0x7f39d4001150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56b12f1e3a5599d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:01.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.655+0000 7f39b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T06:32:01.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.655+0000 7f39b97fa640 1 -- 172.21.15.44:0/3725508931 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d4025020 con 0x55fe14e6bec0 2024-04-22T06:32:01.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.655+0000 7f39b97fa640 1 -- 172.21.15.44:0/3725508931 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f39d4013b40 con 0x55fe14e6bec0 2024-04-22T06:32:01.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.655+0000 7f39e31c0740 1 -- 172.21.15.44:0/3725508931 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39d0055db0 con 0x55fe14e6bec0 2024-04-22T06:32:01.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.655+0000 7f39b97fa640 1 -- 172.21.15.44:0/3725508931 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f39d402f050 con 0x55fe14e6bec0 2024-04-22T06:32:01.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.655+0000 7f39e31c0740 10 client.4941 fetch_fsmap learned FSMap version 18 2024-04-22T06:32:01.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.655+0000 7f39e31c0740 1 -- 172.21.15.44:0/3725508931 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55fe14ebff00 con 0x55fe14e6bec0 2024-04-22T06:32:01.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.655+0000 7f39b97fa640 1 -- 172.21.15.44:0/3725508931 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39d0055db0 con 0x55fe14e6bec0 2024-04-22T06:32:01.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.655+0000 7f39b97fa640 1 -- 172.21.15.44:0/3725508931 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 18) ==== 4792+0+0 (secure 0 0 0) 0x7f39d4032020 con 0x55fe14e6bec0 2024-04-22T06:32:01.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.655+0000 7f39e31c0740 10 client.4941 fetch_fsmap finished waiting for FSMap version 18 2024-04-22T06:32:01.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.655+0000 7f39e31c0740 10 client.4941 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-22T06:32:01.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.655+0000 7f39e31c0740 20 client.4941 populate_metadata read hostname 'smithi044' 2024-04-22T06:32:01.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.655+0000 7f39e31c0740 1 --2- 172.21.15.44:0/3725508931 >> [v2:172.21.15.44:6833/2383621014,v1:172.21.15.44:6835/2383621014] conn(0x55fe14eaf9a0 0x55fe14ecac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.655+0000 7f39e31c0740 4 client.4941 mds_command: new command op to 4290 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T06:32:01.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.655+0000 7f39e31c0740 1 -- 172.21.15.44:0/3725508931 --> [v2:172.21.15.44:6833/2383621014,v1:172.21.15.44:6835/2383621014] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe14ddc530 con 0x55fe14eaf9a0 2024-04-22T06:32:01.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.656+0000 7f39dda8d640 1 --2- 172.21.15.44:0/3725508931 >> [v2:172.21.15.44:6833/2383621014,v1:172.21.15.44:6835/2383621014] conn(0x55fe14eaf9a0 0x55fe14ecac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.656+0000 7f39b97fa640 10 client.4941 ms_handle_connect on v2:172.21.15.44:6833/2383621014 2024-04-22T06:32:01.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.656+0000 7f39dda8d640 1 --2- 172.21.15.44:0/3725508931 >> [v2:172.21.15.44:6833/2383621014,v1:172.21.15.44:6835/2383621014] conn(0x55fe14eaf9a0 0x55fe14ecac70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d96f6fe6607cbba7 server_cookie=71015c33dc94ff31 in_seq=0 out_seq=0 2024-04-22T06:32:01.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.657+0000 7f39b97fa640 1 -- 172.21.15.44:0/3725508931 <== mds.3 v2:172.21.15.44:6833/2383621014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fe14ddc530 con 0x55fe14eaf9a0 2024-04-22T06:32:01.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.657+0000 7f39b97fa640 10 client.4941 handle_command_reply: tid=0 2024-04-22T06:32:01.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.657+0000 7f39e31c0740 1 client.4941 shutdown 2024-04-22T06:32:01.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.657+0000 7f39e31c0740 1 -- 172.21.15.44:0/3725508931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe14e6bec0 msgr2=0x55fe14e68660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:01.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.657+0000 7f39e31c0740 1 --2- 172.21.15.44:0/3725508931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe14e6bec0 0x55fe14e68660 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f39d4014980 tx=0x7f39d4001150 comp rx=0 tx=0).stop 2024-04-22T06:32:01.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.657+0000 7f39e31c0740 1 -- 172.21.15.44:0/3725508931 shutdown_connections 2024-04-22T06:32:01.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.657+0000 7f39e31c0740 1 --2- 172.21.15.44:0/3725508931 >> [v2:172.21.15.44:6833/2383621014,v1:172.21.15.44:6835/2383621014] conn(0x55fe14eaf9a0 0x55fe14ecac70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.657+0000 7f39e31c0740 1 --2- 172.21.15.44:0/3725508931 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fe14ea3890 0x55fe14e877c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.657+0000 7f39e31c0740 1 --2- 172.21.15.44:0/3725508931 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fe14eaae40 0x55fe14ebf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.657+0000 7f39b97fa640 0 client.4941 ms_handle_reset on v2:172.21.15.44:6833/2383621014 2024-04-22T06:32:01.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.657+0000 7f39e31c0740 1 --2- 172.21.15.44:0/3725508931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe14e6bec0 0x55fe14e68660 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.657+0000 7f39e31c0740 1 -- 172.21.15.44:0/3725508931 >> 172.21.15.44:0/3725508931 conn(0x55fe14e67ca0 msgr2=0x55fe14ea9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:01.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.657+0000 7f39e31c0740 1 -- 172.21.15.44:0/3725508931 shutdown_connections 2024-04-22T06:32:01.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.657+0000 7f39e31c0740 1 -- 172.21.15.44:0/3725508931 wait complete. 2024-04-22T06:32:01.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.657+0000 7f39e31c0740 20 client.4941 trim_cache size 0 max 16384 2024-04-22T06:32:01.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.675+0000 7f39e31c0740 1 Processor -- start 2024-04-22T06:32:01.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.675+0000 7f39e31c0740 1 -- start start 2024-04-22T06:32:01.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.676+0000 7f39e31c0740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fe14e6bec0 0x55fe14e69240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.676+0000 7f39e31c0740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fe14ea3890 0x55fe14ed01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.676+0000 7f39e31c0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe14eaae40 0x55fe14ed0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.676+0000 7f39e31c0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f39d81fec60 con 0x55fe14eaae40 2024-04-22T06:32:01.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.676+0000 7f39e31c0740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f39d814df20 con 0x55fe14ea3890 2024-04-22T06:32:01.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.676+0000 7f39e31c0740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55fe14e867a0 con 0x55fe14e6bec0 2024-04-22T06:32:01.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.676+0000 7f39dea8f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe14eaae40 0x55fe14ed0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.676+0000 7f39dea8f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe14eaae40 0x55fe14ed0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55222/0 (socket says 172.21.15.44:55222) 2024-04-22T06:32:01.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.676+0000 7f39dea8f640 1 -- 172.21.15.44:0/1845744065 learned_addr learned my addr 172.21.15.44:0/1845744065 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:01.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.676+0000 7f39de28e640 1 --2- 172.21.15.44:0/1845744065 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fe14e6bec0 0x55fe14e69240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.676+0000 7f39dea8f640 1 -- 172.21.15.44:0/1845744065 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fe14e6bec0 msgr2=0x55fe14e69240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:01.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.676+0000 7f39dea8f640 1 --2- 172.21.15.44:0/1845744065 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fe14e6bec0 0x55fe14e69240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.676+0000 7f39dda8d640 1 --2- 172.21.15.44:0/1845744065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fe14ea3890 0x55fe14ed01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.676+0000 7f39dea8f640 1 -- 172.21.15.44:0/1845744065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fe14ea3890 msgr2=0x55fe14ed01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:01.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.676+0000 7f39dea8f640 1 --2- 172.21.15.44:0/1845744065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fe14ea3890 0x55fe14ed01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.676+0000 7f39dea8f640 1 -- 172.21.15.44:0/1845744065 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe14eab450 con 0x55fe14eaae40 2024-04-22T06:32:01.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.677+0000 7f39dea8f640 1 --2- 172.21.15.44:0/1845744065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe14eaae40 0x55fe14ed0710 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f39d000cfd0 tx=0x7f39d005b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997021713b89283e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:01.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.677+0000 7f39bb7fe640 1 -- 172.21.15.44:0/1845744065 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d0062270 con 0x55fe14eaae40 2024-04-22T06:32:01.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.677+0000 7f39bb7fe640 1 -- 172.21.15.44:0/1845744065 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39d0057a40 con 0x55fe14eaae40 2024-04-22T06:32:01.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.677+0000 7f39bb7fe640 1 -- 172.21.15.44:0/1845744065 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d0061de0 con 0x55fe14eaae40 2024-04-22T06:32:01.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.677+0000 7f39e31c0740 1 -- 172.21.15.44:0/1845744065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe14eaae40 msgr2=0x55fe14ed0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:01.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.677+0000 7f39e31c0740 1 --2- 172.21.15.44:0/1845744065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe14eaae40 0x55fe14ed0710 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f39d000cfd0 tx=0x7f39d005b000 comp rx=0 tx=0).stop 2024-04-22T06:32:01.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.677+0000 7f39e31c0740 1 -- 172.21.15.44:0/1845744065 shutdown_connections 2024-04-22T06:32:01.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.677+0000 7f39e31c0740 1 --2- 172.21.15.44:0/1845744065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe14eaae40 0x55fe14ed0710 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.677+0000 7f39e31c0740 1 --2- 172.21.15.44:0/1845744065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fe14ea3890 0x55fe14ed01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.677+0000 7f39e31c0740 1 --2- 172.21.15.44:0/1845744065 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fe14e6bec0 0x55fe14e69240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.677+0000 7f39e31c0740 1 -- 172.21.15.44:0/1845744065 >> 172.21.15.44:0/1845744065 conn(0x55fe14e67ca0 msgr2=0x55fe14e6a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:01.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.677+0000 7f39e31c0740 1 -- 172.21.15.44:0/1845744065 shutdown_connections 2024-04-22T06:32:01.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.677+0000 7f39e31c0740 1 -- 172.21.15.44:0/1845744065 wait complete. 2024-04-22T06:32:01.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.678+0000 7f39e31c0740 1 -- start start 2024-04-22T06:32:01.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.678+0000 7f39e31c0740 1 Processor -- start 2024-04-22T06:32:01.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.679+0000 7f39e31c0740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fe14e6bec0 0x55fe14e85710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.679+0000 7f39e31c0740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fe14ea3890 0x55fe14e65260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.679+0000 7f39e31c0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe14eaae40 0x55fe14eb1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.679+0000 7f39e31c0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55fe14e87e50 con 0x55fe14eaae40 2024-04-22T06:32:01.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.679+0000 7f39e31c0740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55fe14ea4030 con 0x55fe14e6bec0 2024-04-22T06:32:01.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.679+0000 7f39e31c0740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55fe14eafe70 con 0x55fe14ea3890 2024-04-22T06:32:01.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.679+0000 7f39e31c0740 10 client.0 _ll_register_callbacks cb 0x7f39d40072c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T06:32:01.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.679+0000 7f39dea8f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe14eaae40 0x55fe14eb1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.679+0000 7f39dea8f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe14eaae40 0x55fe14eb1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55232/0 (socket says 172.21.15.44:55232) 2024-04-22T06:32:01.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.679+0000 7f39dea8f640 1 -- 172.21.15.44:0/1411067240 learned_addr learned my addr 172.21.15.44:0/1411067240 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:01.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.679+0000 7f39dda8d640 1 --2- 172.21.15.44:0/1411067240 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fe14ea3890 0x55fe14e65260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.679+0000 7f39de28e640 1 --2- 172.21.15.44:0/1411067240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fe14e6bec0 0x55fe14e85710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.679+0000 7f39dea8f640 1 -- 172.21.15.44:0/1411067240 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fe14ea3890 msgr2=0x55fe14e65260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:01.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.679+0000 7f39dea8f640 1 --2- 172.21.15.44:0/1411067240 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fe14ea3890 0x55fe14e65260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.679+0000 7f39dea8f640 1 -- 172.21.15.44:0/1411067240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fe14e6bec0 msgr2=0x55fe14e85710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:01.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.679+0000 7f39dea8f640 1 --2- 172.21.15.44:0/1411067240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fe14e6bec0 0x55fe14e85710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.679+0000 7f39dea8f640 1 -- 172.21.15.44:0/1411067240 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe14eb22d0 con 0x55fe14eaae40 2024-04-22T06:32:01.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.679+0000 7f39dea8f640 1 --2- 172.21.15.44:0/1411067240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe14eaae40 0x55fe14eb1960 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f39d000c790 tx=0x7f39d0001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5241342524895caa server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:01.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.679+0000 7f39b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T06:32:01.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.679+0000 7f39b97fa640 1 -- 172.21.15.44:0/1411067240 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d0062270 con 0x55fe14eaae40 2024-04-22T06:32:01.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.680+0000 7f39b97fa640 1 -- 172.21.15.44:0/1411067240 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f39d0061de0 con 0x55fe14eaae40 2024-04-22T06:32:01.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.680+0000 7f39e31c0740 1 -- 172.21.15.44:0/1411067240 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39d0057a40 con 0x55fe14eaae40 2024-04-22T06:32:01.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.680+0000 7f39b97fa640 1 -- 172.21.15.44:0/1411067240 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f39d0081050 con 0x55fe14eaae40 2024-04-22T06:32:01.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.680+0000 7f39e31c0740 10 client.4947 fetch_fsmap learned FSMap version 18 2024-04-22T06:32:01.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.680+0000 7f39e31c0740 1 -- 172.21.15.44:0/1411067240 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55fe14f2a390 con 0x55fe14eaae40 2024-04-22T06:32:01.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.680+0000 7f39b97fa640 1 -- 172.21.15.44:0/1411067240 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39d0057a40 con 0x55fe14eaae40 2024-04-22T06:32:01.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.680+0000 7f39b97fa640 1 -- 172.21.15.44:0/1411067240 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 18) ==== 4792+0+0 (secure 0 0 0) 0x7f39d008b040 con 0x55fe14eaae40 2024-04-22T06:32:01.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.680+0000 7f39e31c0740 10 client.4947 fetch_fsmap finished waiting for FSMap version 18 2024-04-22T06:32:01.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.680+0000 7f39e31c0740 10 client.4947 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-22T06:32:01.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.680+0000 7f39e31c0740 20 client.4947 populate_metadata read hostname 'smithi044' 2024-04-22T06:32:01.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.680+0000 7f39e31c0740 1 --2- 172.21.15.44:0/1411067240 >> [v2:172.21.15.44:6833/2383621014,v1:172.21.15.44:6835/2383621014] conn(0x55fe14eadc50 0x55fe14eae030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.680+0000 7f39e31c0740 4 client.4947 mds_command: new command op to 4290 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T06:32:01.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.680+0000 7f39e31c0740 1 -- 172.21.15.44:0/1411067240 --> [v2:172.21.15.44:6833/2383621014,v1:172.21.15.44:6835/2383621014] -- command(tid 0: {"prefix": "status"}) -- 0x55fe14dd63a0 con 0x55fe14eadc50 2024-04-22T06:32:01.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.680+0000 7f39de28e640 1 --2- 172.21.15.44:0/1411067240 >> [v2:172.21.15.44:6833/2383621014,v1:172.21.15.44:6835/2383621014] conn(0x55fe14eadc50 0x55fe14eae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.680+0000 7f39b97fa640 10 client.4947 ms_handle_connect on v2:172.21.15.44:6833/2383621014 2024-04-22T06:32:01.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.680+0000 7f39de28e640 1 --2- 172.21.15.44:0/1411067240 >> [v2:172.21.15.44:6833/2383621014,v1:172.21.15.44:6835/2383621014] conn(0x55fe14eadc50 0x55fe14eae030 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8c6192278200d175 server_cookie=b83ee6acf23bff30 in_seq=0 out_seq=0 2024-04-22T06:32:01.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.681+0000 7f39b97fa640 1 -- 172.21.15.44:0/1411067240 <== mds.3 v2:172.21.15.44:6833/2383621014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55fe14dd63a0 con 0x55fe14eadc50 2024-04-22T06:32:01.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.681+0000 7f39b97fa640 10 client.4947 handle_command_reply: tid=0 2024-04-22T06:32:01.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.681+0000 7f39e31c0740 1 client.4947 shutdown 2024-04-22T06:32:01.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.681+0000 7f39e31c0740 1 -- 172.21.15.44:0/1411067240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe14eaae40 msgr2=0x55fe14eb1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:01.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.681+0000 7f39e31c0740 1 --2- 172.21.15.44:0/1411067240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe14eaae40 0x55fe14eb1960 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f39d000c790 tx=0x7f39d0001250 comp rx=0 tx=0).stop 2024-04-22T06:32:01.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.681+0000 7f39e31c0740 1 -- 172.21.15.44:0/1411067240 shutdown_connections 2024-04-22T06:32:01.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.681+0000 7f39e31c0740 1 --2- 172.21.15.44:0/1411067240 >> [v2:172.21.15.44:6833/2383621014,v1:172.21.15.44:6835/2383621014] conn(0x55fe14eadc50 0x55fe14eae030 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.681+0000 7f39e31c0740 1 --2- 172.21.15.44:0/1411067240 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fe14ea3890 0x55fe14e65260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.681+0000 7f39e31c0740 1 --2- 172.21.15.44:0/1411067240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fe14eaae40 0x55fe14eb1960 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.681+0000 7f39b97fa640 0 client.4947 ms_handle_reset on v2:172.21.15.44:6833/2383621014 2024-04-22T06:32:01.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.681+0000 7f39e31c0740 1 --2- 172.21.15.44:0/1411067240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fe14e6bec0 0x55fe14e85710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.681+0000 7f39e31c0740 1 -- 172.21.15.44:0/1411067240 >> 172.21.15.44:0/1411067240 conn(0x55fe14e67ca0 msgr2=0x55fe14e90620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:01.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.681+0000 7f39e31c0740 1 -- 172.21.15.44:0/1411067240 shutdown_connections 2024-04-22T06:32:01.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.681+0000 7f39e31c0740 1 -- 172.21.15.44:0/1411067240 wait complete. 2024-04-22T06:32:01.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.681+0000 7f39e31c0740 20 client.4947 trim_cache size 0 max 16384 2024-04-22T06:32:01.687 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-22T06:32:01.687 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "9b0a34ca-403d-4513-8a66-d3749ba559aa", 2024-04-22T06:32:01.687 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 3, 2024-04-22T06:32:01.687 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 4290, 2024-04-22T06:32:01.687 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-04-22T06:32:01.687 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-04-22T06:32:01.687 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-04-22T06:32:01.687 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 17.549355535, 2024-04-22T06:32:01.687 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 18, 2024-04-22T06:32:01.687 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 27, 2024-04-22T06:32:01.687 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 27, 2024-04-22T06:32:01.688 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 61.589318517000002 2024-04-22T06:32:01.688 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-22T06:32:01.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.682+0000 7f39e04f2640 1 -- 172.21.15.44:0/927154971 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f39bc06c8f0 msgr2=0x7f39bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:01.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.682+0000 7f39e04f2640 1 --2- 172.21.15.44:0/927154971 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f39bc06c8f0 0x7f39bc06edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f39d40011e0 tx=0x7f39d4007d30 comp rx=0 tx=0).stop 2024-04-22T06:32:01.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.682+0000 7f39e04f2640 1 -- 172.21.15.44:0/927154971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39d81552e0 msgr2=0x7f39d814a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:01.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.682+0000 7f39e04f2640 1 --2- 172.21.15.44:0/927154971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39d81552e0 0x7f39d814a250 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f39d000c7c0 tx=0x7f39d000cc80 comp rx=0 tx=0).stop 2024-04-22T06:32:01.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.682+0000 7f39e04f2640 1 -- 172.21.15.44:0/927154971 shutdown_connections 2024-04-22T06:32:01.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.682+0000 7f39e04f2640 1 --2- 172.21.15.44:0/927154971 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f39bc06c8f0 0x7f39bc06edb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.682+0000 7f39e04f2640 1 --2- 172.21.15.44:0/927154971 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39d8154f00 0x7f39d80cd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.682+0000 7f39e04f2640 1 --2- 172.21.15.44:0/927154971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39d81552e0 0x7f39d814a250 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.682+0000 7f39e04f2640 1 --2- 172.21.15.44:0/927154971 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39d806d470 0x7f39d80cec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.682+0000 7f39e04f2640 1 -- 172.21.15.44:0/927154971 >> 172.21.15.44:0/927154971 conn(0x7f39d80733c0 msgr2=0x7f39d80d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:01.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.682+0000 7f39e04f2640 1 -- 172.21.15.44:0/927154971 shutdown_connections 2024-04-22T06:32:01.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.682+0000 7f39e04f2640 1 -- 172.21.15.44:0/927154971 wait complete. 2024-04-22T06:32:01.691 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-22T06:32:01.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.793+0000 7fa7dea4e640 1 Processor -- start 2024-04-22T06:32:01.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.793+0000 7fa7dea4e640 1 -- start start 2024-04-22T06:32:01.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.794+0000 7fa7dea4e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa7d80c6150 0x7fa7d8158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.794+0000 7fa7dea4e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa7d80c8750 0x7fa7d8070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.794+0000 7fa7dea4e640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7d8073c90 con 0x7fa7d80c5d70 2024-04-22T06:32:01.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.794+0000 7fa7dea4e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fa7d80cf630 con 0x7fa7d80c6150 2024-04-22T06:32:01.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.794+0000 7fa7dea4e640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa7d8073f70 con 0x7fa7d80c8750 2024-04-22T06:32:01.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.794+0000 7fa7d7fff640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fa7d80c5d70 0x7fa7d80731e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40076/0 (socket says 172.21.15.44:40076) 2024-04-22T06:32:01.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.794+0000 7fa7d7fff640 1 -- 172.21.15.44:0/3491330487 learned_addr learned my addr 172.21.15.44:0/3491330487 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:32:01.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.794+0000 7fa7d77fe640 1 --2- 172.21.15.44:0/3491330487 >> v2:172.21.15.70:3300/0 conn(0x7fa7d80c6150 0x7fa7d8158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.794+0000 7fa7dcfeb640 1 --2- 172.21.15.44:0/3491330487 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa7d80c8750 0x7fa7d8070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.795+0000 7fa7d6ffd640 1 -- 172.21.15.44:0/3491330487 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1929336956 0 0) 0x7fa7d8073c90 con 0x7fa7d80c5d70 2024-04-22T06:32:01.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.795+0000 7fa7d6ffd640 1 -- 172.21.15.44:0/3491330487 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa7c0003520 con 0x7fa7d80c5d70 2024-04-22T06:32:01.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.795+0000 7fa7d77fe640 1 -- 172.21.15.44:0/3491330487 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa7d80c8750 msgr2=0x7fa7d8070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:01.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.795+0000 7fa7d77fe640 1 --2- 172.21.15.44:0/3491330487 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa7d80c8750 0x7fa7d8070810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.795+0000 7fa7d77fe640 1 -- 172.21.15.44:0/3491330487 >> v1:172.21.15.44:6789/0 conn(0x7fa7d80c5d70 legacy=0x7fa7d80731e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:01.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.795+0000 7fa7d77fe640 1 -- 172.21.15.44:0/3491330487 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7d80c7c70 con 0x7fa7d80c6150 2024-04-22T06:32:01.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.795+0000 7fa7dcfeb640 1 --2- 172.21.15.44:0/3491330487 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa7d80c8750 0x7fa7d8070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:32:01.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.796+0000 7fa7d77fe640 1 --2- 172.21.15.44:0/3491330487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7d80c6150 0x7fa7d8158ff0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc0097a0 tx=0x7fa7cc02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4abf53fd837bd14 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:01.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.796+0000 7fa7d6ffd640 1 -- 172.21.15.44:0/3491330487 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc034070 con 0x7fa7d80c6150 2024-04-22T06:32:01.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.796+0000 7fa7d6ffd640 1 -- 172.21.15.44:0/3491330487 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7cc030610 con 0x7fa7d80c6150 2024-04-22T06:32:01.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.796+0000 7fa7d6ffd640 1 -- 172.21.15.44:0/3491330487 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc0309e0 con 0x7fa7d80c6150 2024-04-22T06:32:01.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.796+0000 7fa7dea4e640 1 -- 172.21.15.44:0/3491330487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7d80c6150 msgr2=0x7fa7d8158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:01.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.796+0000 7fa7dea4e640 1 --2- 172.21.15.44:0/3491330487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7d80c6150 0x7fa7d8158ff0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc0097a0 tx=0x7fa7cc02b960 comp rx=0 tx=0).stop 2024-04-22T06:32:01.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.796+0000 7fa7dea4e640 1 -- 172.21.15.44:0/3491330487 shutdown_connections 2024-04-22T06:32:01.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.796+0000 7fa7dea4e640 1 --2- 172.21.15.44:0/3491330487 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa7d80c8750 0x7fa7d8070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.796+0000 7fa7dea4e640 1 --2- 172.21.15.44:0/3491330487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7d80c6150 0x7fa7d8158ff0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.796+0000 7fa7dea4e640 1 -- 172.21.15.44:0/3491330487 >> 172.21.15.44:0/3491330487 conn(0x7fa7d80ce700 msgr2=0x7fa7d80ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:01.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.796+0000 7fa7dea4e640 1 -- 172.21.15.44:0/3491330487 shutdown_connections 2024-04-22T06:32:01.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.796+0000 7fa7dea4e640 1 -- 172.21.15.44:0/3491330487 wait complete. 2024-04-22T06:32:01.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.798+0000 7fa7dea4e640 1 Processor -- start 2024-04-22T06:32:01.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.798+0000 7fa7dea4e640 1 -- start start 2024-04-22T06:32:01.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.798+0000 7fa7dea4e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7d80c5d70 0x7fa7d81e15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.798+0000 7fa7dea4e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7d80c6150 0x7fa7d81e26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.798+0000 7fa7dea4e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa7d80c8750 0x7fa7d81e2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.798+0000 7fa7dea4e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d80738f0 con 0x7fa7d80c6150 2024-04-22T06:32:01.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.798+0000 7fa7dea4e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d80c8230 con 0x7fa7d80c5d70 2024-04-22T06:32:01.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.798+0000 7fa7dea4e640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa7d8073a70 con 0x7fa7d80c8750 2024-04-22T06:32:01.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.799+0000 7fa7d77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7d80c6150 0x7fa7d81e26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.799+0000 7fa7d77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7d80c6150 0x7fa7d81e26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52134/0 (socket says 172.21.15.44:52134) 2024-04-22T06:32:01.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.799+0000 7fa7d77fe640 1 -- 172.21.15.44:0/2977140154 learned_addr learned my addr 172.21.15.44:0/2977140154 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:01.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.799+0000 7fa7dcfeb640 1 --2- 172.21.15.44:0/2977140154 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa7d80c8750 0x7fa7d81e2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.799+0000 7fa7d7fff640 1 --2- 172.21.15.44:0/2977140154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7d80c5d70 0x7fa7d81e15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.799+0000 7fa7d77fe640 1 -- 172.21.15.44:0/2977140154 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa7d80c8750 msgr2=0x7fa7d81e2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:01.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.799+0000 7fa7d77fe640 1 --2- 172.21.15.44:0/2977140154 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa7d80c8750 0x7fa7d81e2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.799+0000 7fa7d77fe640 1 -- 172.21.15.44:0/2977140154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7d80c5d70 msgr2=0x7fa7d81e15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:01.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.799+0000 7fa7d77fe640 1 --2- 172.21.15.44:0/2977140154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7d80c5d70 0x7fa7d81e15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.799+0000 7fa7d77fe640 1 -- 172.21.15.44:0/2977140154 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7d81eef10 con 0x7fa7d80c6150 2024-04-22T06:32:01.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.799+0000 7fa7d77fe640 1 --2- 172.21.15.44:0/2977140154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7d80c6150 0x7fa7d81e26b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc009890 tx=0x7fa7cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:01.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.799+0000 7fa7d4ff9640 1 -- 172.21.15.44:0/2977140154 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc034040 con 0x7fa7d80c6150 2024-04-22T06:32:01.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.799+0000 7fa7d4ff9640 1 -- 172.21.15.44:0/2977140154 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7cc030410 con 0x7fa7d80c6150 2024-04-22T06:32:01.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.799+0000 7fa7dea4e640 1 -- 172.21.15.44:0/2977140154 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7d81ef1f0 con 0x7fa7d80c6150 2024-04-22T06:32:01.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.799+0000 7fa7d4ff9640 1 -- 172.21.15.44:0/2977140154 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc006e10 con 0x7fa7d80c6150 2024-04-22T06:32:01.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.799+0000 7fa7dea4e640 1 -- 172.21.15.44:0/2977140154 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7d81ef7b0 con 0x7fa7d80c6150 2024-04-22T06:32:01.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.801+0000 7fa7d4ff9640 1 -- 172.21.15.44:0/2977140154 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa7cc037070 con 0x7fa7d80c6150 2024-04-22T06:32:01.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.801+0000 7fa7d4ff9640 1 --2- 172.21.15.44:0/2977140154 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fa7c406c8f0 0x7fa7c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.801+0000 7fa7d4ff9640 1 -- 172.21.15.44:0/2977140154 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa7cc0ade50 con 0x7fa7d80c6150 2024-04-22T06:32:01.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.802+0000 7fa7d7fff640 1 --2- 172.21.15.44:0/2977140154 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fa7c406c8f0 0x7fa7c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.802+0000 7fa7d7fff640 1 --2- 172.21.15.44:0/2977140154 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fa7c406c8f0 0x7fa7c406edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8005880 tx=0x7fa7c8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:01.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.802+0000 7fa7e1640740 1 Processor -- start 2024-04-22T06:32:01.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.802+0000 7fa7e1640740 1 -- start start 2024-04-22T06:32:01.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.803+0000 7fa7e1640740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab0f507e40 0x55ab0f510f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.803+0000 7fa7e1640740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0f4c8ec0 0x55ab0f5114a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.803+0000 7fa7e1640740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab0f4c4ca0 0x55ab0f4ecee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.803+0000 7fa7e1640740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ab0f42bb10 con 0x55ab0f4c4ca0 2024-04-22T06:32:01.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.803+0000 7fa7e1640740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55ab0f3c9240 con 0x55ab0f4c8ec0 2024-04-22T06:32:01.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.803+0000 7fa7e1640740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55ab0f427ad0 con 0x55ab0f507e40 2024-04-22T06:32:01.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.803+0000 7fa7dcfeb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab0f4c4ca0 0x55ab0f4ecee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.803+0000 7fa7dcfeb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab0f4c4ca0 0x55ab0f4ecee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52150/0 (socket says 172.21.15.44:52150) 2024-04-22T06:32:01.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.803+0000 7fa7d7fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab0f507e40 0x55ab0f510f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.803+0000 7fa7d77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0f4c8ec0 0x55ab0f5114a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.803+0000 7fa7dcfeb640 1 -- 172.21.15.44:0/182555334 learned_addr learned my addr 172.21.15.44:0/182555334 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:01.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.803+0000 7fa7dcfeb640 1 -- 172.21.15.44:0/182555334 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab0f507e40 msgr2=0x55ab0f510f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:01.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.803+0000 7fa7dcfeb640 1 --2- 172.21.15.44:0/182555334 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab0f507e40 0x55ab0f510f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.803+0000 7fa7dcfeb640 1 -- 172.21.15.44:0/182555334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0f4c8ec0 msgr2=0x55ab0f5114a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:01.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.803+0000 7fa7dcfeb640 1 --2- 172.21.15.44:0/182555334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0f4c8ec0 0x55ab0f5114a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.803+0000 7fa7dcfeb640 1 -- 172.21.15.44:0/182555334 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab0f13e2d0 con 0x55ab0f4c4ca0 2024-04-22T06:32:01.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.803+0000 7fa7d77fe640 1 --2- 172.21.15.44:0/182555334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0f4c8ec0 0x55ab0f5114a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:32:01.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.804+0000 7fa7dcfeb640 1 --2- 172.21.15.44:0/182555334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab0f4c4ca0 0x55ab0f4ecee0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa7d00074f0 tx=0x7fa7d0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fea9c27c7bc0b24 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:01.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.804+0000 7fa7bdffb640 1 -- 172.21.15.44:0/182555334 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0019020 con 0x55ab0f4c4ca0 2024-04-22T06:32:01.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.804+0000 7fa7bdffb640 1 -- 172.21.15.44:0/182555334 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7d0015850 con 0x55ab0f4c4ca0 2024-04-22T06:32:01.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.804+0000 7fa7bdffb640 1 -- 172.21.15.44:0/182555334 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0015c00 con 0x55ab0f4c4ca0 2024-04-22T06:32:01.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.804+0000 7fa7e1640740 1 -- 172.21.15.44:0/182555334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab0f4c4ca0 msgr2=0x55ab0f4ecee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:01.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.804+0000 7fa7e1640740 1 --2- 172.21.15.44:0/182555334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab0f4c4ca0 0x55ab0f4ecee0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa7d00074f0 tx=0x7fa7d0009750 comp rx=0 tx=0).stop 2024-04-22T06:32:01.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.804+0000 7fa7e1640740 1 -- 172.21.15.44:0/182555334 shutdown_connections 2024-04-22T06:32:01.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.804+0000 7fa7e1640740 1 --2- 172.21.15.44:0/182555334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab0f4c4ca0 0x55ab0f4ecee0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.804+0000 7fa7e1640740 1 --2- 172.21.15.44:0/182555334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0f4c8ec0 0x55ab0f5114a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.804+0000 7fa7e1640740 1 --2- 172.21.15.44:0/182555334 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab0f507e40 0x55ab0f510f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.804+0000 7fa7e1640740 1 -- 172.21.15.44:0/182555334 >> 172.21.15.44:0/182555334 conn(0x55ab0f4c3990 msgr2=0x55ab0f4c6290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:01.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.804+0000 7fa7e1640740 1 -- 172.21.15.44:0/182555334 shutdown_connections 2024-04-22T06:32:01.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.804+0000 7fa7e1640740 1 -- 172.21.15.44:0/182555334 wait complete. 2024-04-22T06:32:01.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.805+0000 7fa7e1640740 1 -- start start 2024-04-22T06:32:01.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.805+0000 7fa7e1640740 1 Processor -- start 2024-04-22T06:32:01.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.806+0000 7fa7e1640740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab0f4c8ec0 0x55ab0f50f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.806+0000 7fa7e1640740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab0f507e40 0x55ab0f5104d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.806+0000 7fa7e1640740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0f50e910 0x55ab0f50e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.806+0000 7fa7e1640740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ab0f4c7250 con 0x55ab0f4c8ec0 2024-04-22T06:32:01.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.806+0000 7fa7e1640740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55ab0f3b4190 con 0x55ab0f50e910 2024-04-22T06:32:01.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.806+0000 7fa7e1640740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55ab0f4c7530 con 0x55ab0f507e40 2024-04-22T06:32:01.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.806+0000 7fa7e1640740 10 client.0 _ll_register_callbacks cb 0x55ab0f3af8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T06:32:01.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.806+0000 7fa7d7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab0f4c8ec0 0x55ab0f50f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.806+0000 7fa7d7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab0f4c8ec0 0x55ab0f50f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52160/0 (socket says 172.21.15.44:52160) 2024-04-22T06:32:01.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.806+0000 7fa7d7fff640 1 -- 172.21.15.44:0/235356126 learned_addr learned my addr 172.21.15.44:0/235356126 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:01.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.806+0000 7fa7dcfeb640 1 --2- 172.21.15.44:0/235356126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0f50e910 0x55ab0f50e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.806+0000 7fa7d77fe640 1 --2- 172.21.15.44:0/235356126 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab0f507e40 0x55ab0f5104d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.806+0000 7fa7d7fff640 1 -- 172.21.15.44:0/235356126 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab0f507e40 msgr2=0x55ab0f5104d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:01.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.806+0000 7fa7d7fff640 1 --2- 172.21.15.44:0/235356126 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab0f507e40 0x55ab0f5104d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.806+0000 7fa7d7fff640 1 -- 172.21.15.44:0/235356126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0f50e910 msgr2=0x55ab0f50e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:01.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.806+0000 7fa7d7fff640 1 --2- 172.21.15.44:0/235356126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0f50e910 0x55ab0f50e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.806+0000 7fa7d7fff640 1 -- 172.21.15.44:0/235356126 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab0f2086b0 con 0x55ab0f4c8ec0 2024-04-22T06:32:01.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.806+0000 7fa7dcfeb640 1 --2- 172.21.15.44:0/235356126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0f50e910 0x55ab0f50e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:32:01.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.807+0000 7fa7d7fff640 1 --2- 172.21.15.44:0/235356126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab0f4c8ec0 0x55ab0f50f830 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa7c80070d0 tx=0x7fa7c80010a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45287f40b6d417ae server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:01.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.807+0000 7fa7b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T06:32:01.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.807+0000 7fa7b6ffd640 1 -- 172.21.15.44:0/235356126 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8026060 con 0x55ab0f4c8ec0 2024-04-22T06:32:01.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.807+0000 7fa7b6ffd640 1 -- 172.21.15.44:0/235356126 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa7c8010040 con 0x55ab0f4c8ec0 2024-04-22T06:32:01.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.807+0000 7fa7e1640740 1 -- 172.21.15.44:0/235356126 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7d0015850 con 0x55ab0f4c8ec0 2024-04-22T06:32:01.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.807+0000 7fa7b6ffd640 1 -- 172.21.15.44:0/235356126 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fa7c802c050 con 0x55ab0f4c8ec0 2024-04-22T06:32:01.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.807+0000 7fa7e1640740 10 client.4959 fetch_fsmap learned FSMap version 18 2024-04-22T06:32:01.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.807+0000 7fa7e1640740 1 -- 172.21.15.44:0/235356126 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55ab0f4c0e70 con 0x55ab0f4c8ec0 2024-04-22T06:32:01.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.807+0000 7fa7b6ffd640 1 -- 172.21.15.44:0/235356126 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7d0015850 con 0x55ab0f4c8ec0 2024-04-22T06:32:01.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.807+0000 7fa7b6ffd640 1 -- 172.21.15.44:0/235356126 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 18) ==== 4792+0+0 (secure 0 0 0) 0x7fa7c802f020 con 0x55ab0f4c8ec0 2024-04-22T06:32:01.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.807+0000 7fa7e1640740 10 client.4959 fetch_fsmap finished waiting for FSMap version 18 2024-04-22T06:32:01.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.807+0000 7fa7e1640740 10 client.4959 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-22T06:32:01.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.807+0000 7fa7e1640740 20 client.4959 populate_metadata read hostname 'smithi044' 2024-04-22T06:32:01.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.807+0000 7fa7e1640740 1 --2- 172.21.15.44:0/235356126 >> [v2:172.21.15.44:6836/915960587,v1:172.21.15.44:6837/915960587] conn(0x55ab0f525b60 0x55ab0f527f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.807+0000 7fa7e1640740 4 client.4959 mds_command: new command op to 4293 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T06:32:01.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.807+0000 7fa7e1640740 1 -- 172.21.15.44:0/235356126 --> [v2:172.21.15.44:6836/915960587,v1:172.21.15.44:6837/915960587] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab0f439530 con 0x55ab0f525b60 2024-04-22T06:32:01.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.807+0000 7fa7d77fe640 1 --2- 172.21.15.44:0/235356126 >> [v2:172.21.15.44:6836/915960587,v1:172.21.15.44:6837/915960587] conn(0x55ab0f525b60 0x55ab0f527f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.808+0000 7fa7b6ffd640 10 client.4959 ms_handle_connect on v2:172.21.15.44:6836/915960587 2024-04-22T06:32:01.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.808+0000 7fa7d77fe640 1 --2- 172.21.15.44:0/235356126 >> [v2:172.21.15.44:6836/915960587,v1:172.21.15.44:6837/915960587] conn(0x55ab0f525b60 0x55ab0f527f50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b6ea0b7a23898814 server_cookie=1dbbfc694a815326 in_seq=0 out_seq=0 2024-04-22T06:32:01.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.808+0000 7fa7b6ffd640 1 -- 172.21.15.44:0/235356126 <== mds.2 v2:172.21.15.44:6836/915960587 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ab0f439530 con 0x55ab0f525b60 2024-04-22T06:32:01.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.808+0000 7fa7b6ffd640 10 client.4959 handle_command_reply: tid=0 2024-04-22T06:32:01.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.808+0000 7fa7e1640740 1 client.4959 shutdown 2024-04-22T06:32:01.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.808+0000 7fa7e1640740 1 -- 172.21.15.44:0/235356126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab0f4c8ec0 msgr2=0x55ab0f50f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:01.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.808+0000 7fa7e1640740 1 --2- 172.21.15.44:0/235356126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab0f4c8ec0 0x55ab0f50f830 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa7c80070d0 tx=0x7fa7c80010a0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.809+0000 7fa7e1640740 1 -- 172.21.15.44:0/235356126 shutdown_connections 2024-04-22T06:32:01.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.809+0000 7fa7e1640740 1 --2- 172.21.15.44:0/235356126 >> [v2:172.21.15.44:6836/915960587,v1:172.21.15.44:6837/915960587] conn(0x55ab0f525b60 0x55ab0f527f50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.809+0000 7fa7e1640740 1 --2- 172.21.15.44:0/235356126 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab0f507e40 0x55ab0f5104d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.809+0000 7fa7e1640740 1 --2- 172.21.15.44:0/235356126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0f50e910 0x55ab0f50e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.809+0000 7fa7b6ffd640 0 client.4959 ms_handle_reset on v2:172.21.15.44:6836/915960587 2024-04-22T06:32:01.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.809+0000 7fa7e1640740 1 --2- 172.21.15.44:0/235356126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab0f4c8ec0 0x55ab0f50f830 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.809+0000 7fa7e1640740 1 -- 172.21.15.44:0/235356126 >> 172.21.15.44:0/235356126 conn(0x55ab0f4c4ca0 msgr2=0x55ab0f4c7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:01.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.809+0000 7fa7e1640740 1 -- 172.21.15.44:0/235356126 shutdown_connections 2024-04-22T06:32:01.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.809+0000 7fa7e1640740 1 -- 172.21.15.44:0/235356126 wait complete. 2024-04-22T06:32:01.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.809+0000 7fa7e1640740 20 client.4959 trim_cache size 0 max 16384 2024-04-22T06:32:01.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.827+0000 7fa7e1640740 1 Processor -- start 2024-04-22T06:32:01.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.827+0000 7fa7e1640740 1 -- start start 2024-04-22T06:32:01.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.828+0000 7fa7e1640740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab0f4c8ec0 0x55ab0f50b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.828+0000 7fa7e1640740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0f507e40 0x55ab0f507130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.828+0000 7fa7e1640740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab0f50e910 0x55ab0f507670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.828+0000 7fa7e1640740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d81fc560 con 0x55ab0f50e910 2024-04-22T06:32:01.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.828+0000 7fa7e1640740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d81fcc30 con 0x55ab0f507e40 2024-04-22T06:32:01.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.828+0000 7fa7e1640740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa7d814f7d0 con 0x55ab0f4c8ec0 2024-04-22T06:32:01.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.828+0000 7fa7dcfeb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab0f50e910 0x55ab0f507670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.828+0000 7fa7dcfeb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab0f50e910 0x55ab0f507670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52172/0 (socket says 172.21.15.44:52172) 2024-04-22T06:32:01.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.828+0000 7fa7dcfeb640 1 -- 172.21.15.44:0/1419915152 learned_addr learned my addr 172.21.15.44:0/1419915152 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:01.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.828+0000 7fa7d77fe640 1 --2- 172.21.15.44:0/1419915152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0f507e40 0x55ab0f507130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.828+0000 7fa7d7fff640 1 --2- 172.21.15.44:0/1419915152 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab0f4c8ec0 0x55ab0f50b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.828+0000 7fa7dcfeb640 1 -- 172.21.15.44:0/1419915152 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab0f4c8ec0 msgr2=0x55ab0f50b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:01.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.828+0000 7fa7dcfeb640 1 --2- 172.21.15.44:0/1419915152 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab0f4c8ec0 0x55ab0f50b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.828+0000 7fa7dcfeb640 1 -- 172.21.15.44:0/1419915152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0f507e40 msgr2=0x55ab0f507130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:01.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.828+0000 7fa7dcfeb640 1 --2- 172.21.15.44:0/1419915152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0f507e40 0x55ab0f507130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.828+0000 7fa7dcfeb640 1 -- 172.21.15.44:0/1419915152 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab0f502260 con 0x55ab0f50e910 2024-04-22T06:32:01.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.828+0000 7fa7d77fe640 1 --2- 172.21.15.44:0/1419915152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0f507e40 0x55ab0f507130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:32:01.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.828+0000 7fa7dcfeb640 1 --2- 172.21.15.44:0/1419915152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab0f50e910 0x55ab0f507670 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0007e40 tx=0x7fa7d000e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3fecf396bb61f89 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:01.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.828+0000 7fa7bd7fa640 1 -- 172.21.15.44:0/1419915152 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0019040 con 0x55ab0f50e910 2024-04-22T06:32:01.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.828+0000 7fa7bd7fa640 1 -- 172.21.15.44:0/1419915152 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7d0003990 con 0x55ab0f50e910 2024-04-22T06:32:01.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.828+0000 7fa7bd7fa640 1 -- 172.21.15.44:0/1419915152 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0003d80 con 0x55ab0f50e910 2024-04-22T06:32:01.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.829+0000 7fa7e1640740 1 -- 172.21.15.44:0/1419915152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab0f50e910 msgr2=0x55ab0f507670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:01.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.829+0000 7fa7e1640740 1 --2- 172.21.15.44:0/1419915152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab0f50e910 0x55ab0f507670 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0007e40 tx=0x7fa7d000e000 comp rx=0 tx=0).stop 2024-04-22T06:32:01.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.829+0000 7fa7e1640740 1 -- 172.21.15.44:0/1419915152 shutdown_connections 2024-04-22T06:32:01.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.829+0000 7fa7e1640740 1 --2- 172.21.15.44:0/1419915152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab0f50e910 0x55ab0f507670 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.829+0000 7fa7e1640740 1 --2- 172.21.15.44:0/1419915152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0f507e40 0x55ab0f507130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.829+0000 7fa7e1640740 1 --2- 172.21.15.44:0/1419915152 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab0f4c8ec0 0x55ab0f50b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.829+0000 7fa7e1640740 1 -- 172.21.15.44:0/1419915152 >> 172.21.15.44:0/1419915152 conn(0x55ab0f4c4ca0 msgr2=0x55ab0f4c2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:01.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.829+0000 7fa7e1640740 1 -- 172.21.15.44:0/1419915152 shutdown_connections 2024-04-22T06:32:01.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.829+0000 7fa7e1640740 1 -- 172.21.15.44:0/1419915152 wait complete. 2024-04-22T06:32:01.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.830+0000 7fa7e1640740 1 -- start start 2024-04-22T06:32:01.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.830+0000 7fa7e1640740 1 Processor -- start 2024-04-22T06:32:01.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.830+0000 7fa7e1640740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0f4c8ec0 0x55ab0f58b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.830+0000 7fa7e1640740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab0f507e40 0x55ab0f507490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.830+0000 7fa7e1640740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab0f50e910 0x55ab0f52d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.830+0000 7fa7e1640740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ab0f4c1f20 con 0x55ab0f507e40 2024-04-22T06:32:01.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.830+0000 7fa7e1640740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55ab0f4c01d0 con 0x55ab0f4c8ec0 2024-04-22T06:32:01.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.830+0000 7fa7e1640740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55ab0f5018a0 con 0x55ab0f50e910 2024-04-22T06:32:01.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.831+0000 7fa7e1640740 10 client.0 _ll_register_callbacks cb 0x7fa7c8001de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T06:32:01.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.831+0000 7fa7d77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab0f507e40 0x55ab0f507490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.831+0000 7fa7d77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab0f507e40 0x55ab0f507490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52182/0 (socket says 172.21.15.44:52182) 2024-04-22T06:32:01.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.831+0000 7fa7d77fe640 1 -- 172.21.15.44:0/2493979465 learned_addr learned my addr 172.21.15.44:0/2493979465 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:01.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.831+0000 7fa7dcfeb640 1 --2- 172.21.15.44:0/2493979465 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab0f50e910 0x55ab0f52d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.831+0000 7fa7d7fff640 1 --2- 172.21.15.44:0/2493979465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0f4c8ec0 0x55ab0f58b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.831+0000 7fa7d77fe640 1 -- 172.21.15.44:0/2493979465 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab0f50e910 msgr2=0x55ab0f52d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:01.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.831+0000 7fa7d77fe640 1 --2- 172.21.15.44:0/2493979465 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab0f50e910 0x55ab0f52d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.831+0000 7fa7d77fe640 1 -- 172.21.15.44:0/2493979465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0f4c8ec0 msgr2=0x55ab0f58b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:01.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.831+0000 7fa7d77fe640 1 --2- 172.21.15.44:0/2493979465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0f4c8ec0 0x55ab0f58b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.831+0000 7fa7d77fe640 1 -- 172.21.15.44:0/2493979465 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab0f58aa00 con 0x55ab0f507e40 2024-04-22T06:32:01.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.831+0000 7fa7d7fff640 1 --2- 172.21.15.44:0/2493979465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0f4c8ec0 0x55ab0f58b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:32:01.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.831+0000 7fa7dcfeb640 1 --2- 172.21.15.44:0/2493979465 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab0f50e910 0x55ab0f52d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:32:01.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.831+0000 7fa7d77fe640 1 --2- 172.21.15.44:0/2493979465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab0f507e40 0x55ab0f507490 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc039680 tx=0x7fa7cc009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e8161b8e66a4832 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:01.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.831+0000 7fa7b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T06:32:01.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.831+0000 7fa7b6ffd640 1 -- 172.21.15.44:0/2493979465 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc084350 con 0x55ab0f507e40 2024-04-22T06:32:01.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.831+0000 7fa7b6ffd640 1 -- 172.21.15.44:0/2493979465 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa7cc079a70 con 0x55ab0f507e40 2024-04-22T06:32:01.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.831+0000 7fa7e1640740 1 -- 172.21.15.44:0/2493979465 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7d0003990 con 0x55ab0f507e40 2024-04-22T06:32:01.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.831+0000 7fa7b6ffd640 1 -- 172.21.15.44:0/2493979465 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fa7cc077de0 con 0x55ab0f507e40 2024-04-22T06:32:01.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.831+0000 7fa7e1640740 10 client.4965 fetch_fsmap learned FSMap version 18 2024-04-22T06:32:01.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.831+0000 7fa7e1640740 1 -- 172.21.15.44:0/2493979465 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55ab0f58aca0 con 0x55ab0f507e40 2024-04-22T06:32:01.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.831+0000 7fa7b6ffd640 1 -- 172.21.15.44:0/2493979465 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7d0003990 con 0x55ab0f507e40 2024-04-22T06:32:01.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.832+0000 7fa7b6ffd640 1 -- 172.21.15.44:0/2493979465 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 18) ==== 4792+0+0 (secure 0 0 0) 0x7fa7cc041070 con 0x55ab0f507e40 2024-04-22T06:32:01.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.832+0000 7fa7e1640740 10 client.4965 fetch_fsmap finished waiting for FSMap version 18 2024-04-22T06:32:01.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.832+0000 7fa7e1640740 10 client.4965 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-22T06:32:01.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.832+0000 7fa7e1640740 20 client.4965 populate_metadata read hostname 'smithi044' 2024-04-22T06:32:01.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.832+0000 7fa7e1640740 1 --2- 172.21.15.44:0/2493979465 >> [v2:172.21.15.44:6836/915960587,v1:172.21.15.44:6837/915960587] conn(0x55ab0f4ebe40 0x55ab0f593d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.832+0000 7fa7e1640740 4 client.4965 mds_command: new command op to 4293 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T06:32:01.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.832+0000 7fa7e1640740 1 -- 172.21.15.44:0/2493979465 --> [v2:172.21.15.44:6836/915960587,v1:172.21.15.44:6837/915960587] -- command(tid 0: {"prefix": "status"}) -- 0x55ab0f4333a0 con 0x55ab0f4ebe40 2024-04-22T06:32:01.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.832+0000 7fa7d7fff640 1 --2- 172.21.15.44:0/2493979465 >> [v2:172.21.15.44:6836/915960587,v1:172.21.15.44:6837/915960587] conn(0x55ab0f4ebe40 0x55ab0f593d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.832+0000 7fa7d7fff640 1 --2- 172.21.15.44:0/2493979465 >> [v2:172.21.15.44:6836/915960587,v1:172.21.15.44:6837/915960587] conn(0x55ab0f4ebe40 0x55ab0f593d40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b19faa150c32ac75 server_cookie=4817dfc90e6bdba6 in_seq=0 out_seq=0 2024-04-22T06:32:01.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.832+0000 7fa7b6ffd640 10 client.4965 ms_handle_connect on v2:172.21.15.44:6836/915960587 2024-04-22T06:32:01.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.832+0000 7fa7b6ffd640 1 -- 172.21.15.44:0/2493979465 <== mds.2 v2:172.21.15.44:6836/915960587 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55ab0f4333a0 con 0x55ab0f4ebe40 2024-04-22T06:32:01.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.832+0000 7fa7b6ffd640 10 client.4965 handle_command_reply: tid=0 2024-04-22T06:32:01.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.832+0000 7fa7e1640740 1 client.4965 shutdown 2024-04-22T06:32:01.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.832+0000 7fa7e1640740 1 -- 172.21.15.44:0/2493979465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab0f507e40 msgr2=0x55ab0f507490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:01.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.832+0000 7fa7e1640740 1 --2- 172.21.15.44:0/2493979465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab0f507e40 0x55ab0f507490 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc039680 tx=0x7fa7cc009190 comp rx=0 tx=0).stop 2024-04-22T06:32:01.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.832+0000 7fa7e1640740 1 -- 172.21.15.44:0/2493979465 shutdown_connections 2024-04-22T06:32:01.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.832+0000 7fa7e1640740 1 --2- 172.21.15.44:0/2493979465 >> [v2:172.21.15.44:6836/915960587,v1:172.21.15.44:6837/915960587] conn(0x55ab0f4ebe40 0x55ab0f593d40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.832+0000 7fa7e1640740 1 --2- 172.21.15.44:0/2493979465 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab0f50e910 0x55ab0f52d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.832+0000 7fa7e1640740 1 --2- 172.21.15.44:0/2493979465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab0f507e40 0x55ab0f507490 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.832+0000 7fa7e1640740 1 --2- 172.21.15.44:0/2493979465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0f4c8ec0 0x55ab0f58b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.832+0000 7fa7b6ffd640 0 client.4965 ms_handle_reset on v2:172.21.15.44:6836/915960587 2024-04-22T06:32:01.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.832+0000 7fa7e1640740 1 -- 172.21.15.44:0/2493979465 >> 172.21.15.44:0/2493979465 conn(0x55ab0f4c4ca0 msgr2=0x55ab0f50b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:01.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.833+0000 7fa7e1640740 1 -- 172.21.15.44:0/2493979465 shutdown_connections 2024-04-22T06:32:01.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.833+0000 7fa7e1640740 1 -- 172.21.15.44:0/2493979465 wait complete. 2024-04-22T06:32:01.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.833+0000 7fa7e1640740 20 client.4965 trim_cache size 0 max 16384 2024-04-22T06:32:01.839 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-22T06:32:01.839 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "9b0a34ca-403d-4513-8a66-d3749ba559aa", 2024-04-22T06:32:01.839 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 2, 2024-04-22T06:32:01.839 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 4293, 2024-04-22T06:32:01.839 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-04-22T06:32:01.839 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-04-22T06:32:01.839 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-04-22T06:32:01.839 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 23.724036439999999, 2024-04-22T06:32:01.839 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 18, 2024-04-22T06:32:01.839 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 27, 2024-04-22T06:32:01.839 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 27, 2024-04-22T06:32:01.840 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 61.734737879999997 2024-04-22T06:32:01.840 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-22T06:32:01.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.833+0000 7fa7dea4e640 1 -- 172.21.15.44:0/2977140154 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fa7c406c8f0 msgr2=0x7fa7c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:01.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.833+0000 7fa7dea4e640 1 --2- 172.21.15.44:0/2977140154 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fa7c406c8f0 0x7fa7c406edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8005880 tx=0x7fa7c8009040 comp rx=0 tx=0).stop 2024-04-22T06:32:01.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.833+0000 7fa7dea4e640 1 -- 172.21.15.44:0/2977140154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7d80c6150 msgr2=0x7fa7d81e26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:01.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.833+0000 7fa7dea4e640 1 --2- 172.21.15.44:0/2977140154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7d80c6150 0x7fa7d81e26b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc009890 tx=0x7fa7cc0046c0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.833+0000 7fa7dea4e640 1 -- 172.21.15.44:0/2977140154 shutdown_connections 2024-04-22T06:32:01.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.833+0000 7fa7dea4e640 1 --2- 172.21.15.44:0/2977140154 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fa7c406c8f0 0x7fa7c406edb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.833+0000 7fa7dea4e640 1 --2- 172.21.15.44:0/2977140154 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa7d80c8750 0x7fa7d81e2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.833+0000 7fa7dea4e640 1 --2- 172.21.15.44:0/2977140154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7d80c6150 0x7fa7d81e26b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.833+0000 7fa7dea4e640 1 --2- 172.21.15.44:0/2977140154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7d80c5d70 0x7fa7d81e15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.833+0000 7fa7dea4e640 1 -- 172.21.15.44:0/2977140154 >> 172.21.15.44:0/2977140154 conn(0x7fa7d80ce700 msgr2=0x7fa7d80ceae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:01.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.833+0000 7fa7dea4e640 1 -- 172.21.15.44:0/2977140154 shutdown_connections 2024-04-22T06:32:01.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.833+0000 7fa7dea4e640 1 -- 172.21.15.44:0/2977140154 wait complete. 2024-04-22T06:32:01.843 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-22T06:32:01.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.945+0000 7fc5762ba640 1 Processor -- start 2024-04-22T06:32:01.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.945+0000 7fc5762ba640 1 -- start start 2024-04-22T06:32:01.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.946+0000 7fc5762ba640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc57006fa80 0x7fc57015bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.946+0000 7fc5762ba640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc57006fe60 0x7fc5700705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.946+0000 7fc5762ba640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5700c7fb0 con 0x7fc570151c80 2024-04-22T06:32:01.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.946+0000 7fc5762ba640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fc5700c8250 con 0x7fc57006fe60 2024-04-22T06:32:01.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.946+0000 7fc5762ba640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fc5700c8530 con 0x7fc57006fa80 2024-04-22T06:32:01.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.946+0000 7fc56f7fe640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc57006fe60 0x7fc5700705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.946+0000 7fc574857640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fc570151c80 0x7fc570152090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40080/0 (socket says 172.21.15.44:40080) 2024-04-22T06:32:01.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.946+0000 7fc574857640 1 -- 172.21.15.44:0/677437944 learned_addr learned my addr 172.21.15.44:0/677437944 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:32:01.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.946+0000 7fc56ffff640 1 --2- 172.21.15.44:0/677437944 >> v2:172.21.15.70:3300/0 conn(0x7fc57006fa80 0x7fc57015bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.947+0000 7fc56f7fe640 1 -- 172.21.15.44:0/677437944 >> v2:172.21.15.70:3300/0 conn(0x7fc57006fa80 msgr2=0x7fc57015bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:01.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.947+0000 7fc56f7fe640 1 --2- 172.21.15.44:0/677437944 >> v2:172.21.15.70:3300/0 conn(0x7fc57006fa80 0x7fc57015bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.947+0000 7fc56f7fe640 1 -- 172.21.15.44:0/677437944 >> v1:172.21.15.44:6789/0 conn(0x7fc570151c80 legacy=0x7fc570152090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:01.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.947+0000 7fc56f7fe640 1 -- 172.21.15.44:0/677437944 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5700c8810 con 0x7fc57006fe60 2024-04-22T06:32:01.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.947+0000 7fc56effd640 1 -- 172.21.15.44:0/677437944 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1982982949 0 0) 0x7fc5700c7fb0 con 0x7fc570151c80 2024-04-22T06:32:01.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.947+0000 7fc56f7fe640 1 --2- 172.21.15.44:0/677437944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc57006fe60 0x7fc5700705c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc570070dd0 tx=0x7fc56402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2dc9c9117e63992 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:01.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.948+0000 7fc56effd640 1 -- 172.21.15.44:0/677437944 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc564034070 con 0x7fc57006fe60 2024-04-22T06:32:01.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.948+0000 7fc56effd640 1 -- 172.21.15.44:0/677437944 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc564030610 con 0x7fc57006fe60 2024-04-22T06:32:01.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.948+0000 7fc56effd640 1 -- 172.21.15.44:0/677437944 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5640309e0 con 0x7fc57006fe60 2024-04-22T06:32:01.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.948+0000 7fc5762ba640 1 -- 172.21.15.44:0/677437944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc57006fe60 msgr2=0x7fc5700705c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:01.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.948+0000 7fc5762ba640 1 --2- 172.21.15.44:0/677437944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc57006fe60 0x7fc5700705c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc570070dd0 tx=0x7fc56402b960 comp rx=0 tx=0).stop 2024-04-22T06:32:01.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.948+0000 7fc5762ba640 1 -- 172.21.15.44:0/677437944 shutdown_connections 2024-04-22T06:32:01.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.948+0000 7fc5762ba640 1 --2- 172.21.15.44:0/677437944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc57006fe60 0x7fc5700705c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.948+0000 7fc5762ba640 1 --2- 172.21.15.44:0/677437944 >> v2:172.21.15.70:3300/0 conn(0x7fc57006fa80 0x7fc57015bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.948+0000 7fc5762ba640 1 -- 172.21.15.44:0/677437944 >> 172.21.15.44:0/677437944 conn(0x7fc5700cd630 msgr2=0x7fc5700cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:01.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.948+0000 7fc5762ba640 1 -- 172.21.15.44:0/677437944 shutdown_connections 2024-04-22T06:32:01.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.948+0000 7fc5762ba640 1 -- 172.21.15.44:0/677437944 wait complete. 2024-04-22T06:32:01.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.949+0000 7fc5762ba640 1 Processor -- start 2024-04-22T06:32:01.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.949+0000 7fc5762ba640 1 -- start start 2024-04-22T06:32:01.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.950+0000 7fc5762ba640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc57006fa80 0x7fc5701ef2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.950+0000 7fc5762ba640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc57006fe60 0x7fc5701538d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.950+0000 7fc5762ba640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc570151c80 0x7fc570152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.950+0000 7fc5762ba640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc5700c90b0 con 0x7fc57006fa80 2024-04-22T06:32:01.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.950+0000 7fc5762ba640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc5700c8dd0 con 0x7fc570151c80 2024-04-22T06:32:01.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.950+0000 7fc5762ba640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc570070b00 con 0x7fc57006fe60 2024-04-22T06:32:01.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.950+0000 7fc56ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc57006fa80 0x7fc5701ef2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.950+0000 7fc56ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc57006fa80 0x7fc5701ef2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52194/0 (socket says 172.21.15.44:52194) 2024-04-22T06:32:01.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.950+0000 7fc56ffff640 1 -- 172.21.15.44:0/2979005756 learned_addr learned my addr 172.21.15.44:0/2979005756 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:01.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.950+0000 7fc574857640 1 --2- 172.21.15.44:0/2979005756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc570151c80 0x7fc570152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.950+0000 7fc56f7fe640 1 --2- 172.21.15.44:0/2979005756 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc57006fe60 0x7fc5701538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.951+0000 7fc56ffff640 1 -- 172.21.15.44:0/2979005756 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc57006fe60 msgr2=0x7fc5701538d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:01.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.951+0000 7fc56ffff640 1 --2- 172.21.15.44:0/2979005756 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc57006fe60 0x7fc5701538d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.951+0000 7fc56ffff640 1 -- 172.21.15.44:0/2979005756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc570151c80 msgr2=0x7fc570152a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:01.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.951+0000 7fc56ffff640 1 --2- 172.21.15.44:0/2979005756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc570151c80 0x7fc570152a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.951+0000 7fc56ffff640 1 -- 172.21.15.44:0/2979005756 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc570074130 con 0x7fc57006fa80 2024-04-22T06:32:01.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.951+0000 7fc56ffff640 1 --2- 172.21.15.44:0/2979005756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc57006fa80 0x7fc5701ef2e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc56000e8a0 tx=0x7fc56000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:01.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.951+0000 7fc56cff9640 1 -- 172.21.15.44:0/2979005756 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc560017070 con 0x7fc57006fa80 2024-04-22T06:32:01.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.951+0000 7fc56cff9640 1 -- 172.21.15.44:0/2979005756 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc56000c940 con 0x7fc57006fa80 2024-04-22T06:32:01.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.951+0000 7fc5762ba640 1 -- 172.21.15.44:0/2979005756 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5701e2930 con 0x7fc57006fa80 2024-04-22T06:32:01.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.951+0000 7fc56cff9640 1 -- 172.21.15.44:0/2979005756 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56000cd30 con 0x7fc57006fa80 2024-04-22T06:32:01.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.951+0000 7fc5762ba640 1 -- 172.21.15.44:0/2979005756 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5701e2ef0 con 0x7fc57006fa80 2024-04-22T06:32:01.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.953+0000 7fc56cff9640 1 -- 172.21.15.44:0/2979005756 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc56001a070 con 0x7fc57006fa80 2024-04-22T06:32:01.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.953+0000 7fc56cff9640 1 --2- 172.21.15.44:0/2979005756 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fc55c06c8f0 0x7fc55c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.953+0000 7fc56cff9640 1 -- 172.21.15.44:0/2979005756 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc56008bd80 con 0x7fc57006fa80 2024-04-22T06:32:01.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.953+0000 7fc56f7fe640 1 --2- 172.21.15.44:0/2979005756 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fc55c06c8f0 0x7fc55c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.954+0000 7fc56f7fe640 1 --2- 172.21.15.44:0/2979005756 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fc55c06c8f0 0x7fc55c06edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc564036000 tx=0x7fc56403b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:01.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.954+0000 7fc578f89740 1 Processor -- start 2024-04-22T06:32:01.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.954+0000 7fc578f89740 1 -- start start 2024-04-22T06:32:01.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.954+0000 7fc578f89740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7747be40 0x55cc77484f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.954+0000 7fc578f89740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc7743cec0 0x55cc774854a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.954+0000 7fc578f89740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc77438ca0 0x55cc77460ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.954+0000 7fc578f89740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55cc7739fb10 con 0x55cc7747be40 2024-04-22T06:32:01.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.954+0000 7fc578f89740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55cc7733d240 con 0x55cc77438ca0 2024-04-22T06:32:01.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.954+0000 7fc578f89740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55cc7739bad0 con 0x55cc7743cec0 2024-04-22T06:32:01.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.955+0000 7fc56ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7747be40 0x55cc77484f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.955+0000 7fc56ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7747be40 0x55cc77484f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52196/0 (socket says 172.21.15.44:52196) 2024-04-22T06:32:01.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.955+0000 7fc56ffff640 1 -- 172.21.15.44:0/3636643983 learned_addr learned my addr 172.21.15.44:0/3636643983 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:01.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.955+0000 7fc56f7fe640 1 --2- 172.21.15.44:0/3636643983 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc7743cec0 0x55cc774854a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.955+0000 7fc574857640 1 --2- 172.21.15.44:0/3636643983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc77438ca0 0x55cc77460ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.955+0000 7fc56ffff640 1 -- 172.21.15.44:0/3636643983 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc7743cec0 msgr2=0x55cc774854a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:01.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.955+0000 7fc56ffff640 1 --2- 172.21.15.44:0/3636643983 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc7743cec0 0x55cc774854a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.955+0000 7fc56ffff640 1 -- 172.21.15.44:0/3636643983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc77438ca0 msgr2=0x55cc77460ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:01.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.955+0000 7fc56ffff640 1 --2- 172.21.15.44:0/3636643983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc77438ca0 0x55cc77460ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.955+0000 7fc56ffff640 1 -- 172.21.15.44:0/3636643983 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc770b22d0 con 0x55cc7747be40 2024-04-22T06:32:01.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.955+0000 7fc56ffff640 1 --2- 172.21.15.44:0/3636643983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7747be40 0x55cc77484f60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc560061d60 tx=0x7fc560059030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=381a12460ee3d714 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:01.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.955+0000 7fc555ffb640 1 -- 172.21.15.44:0/3636643983 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc560088070 con 0x55cc7747be40 2024-04-22T06:32:01.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.955+0000 7fc555ffb640 1 -- 172.21.15.44:0/3636643983 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc560062290 con 0x55cc7747be40 2024-04-22T06:32:01.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.955+0000 7fc555ffb640 1 -- 172.21.15.44:0/3636643983 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc560021040 con 0x55cc7747be40 2024-04-22T06:32:01.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.956+0000 7fc578f89740 1 -- 172.21.15.44:0/3636643983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7747be40 msgr2=0x55cc77484f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:01.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.956+0000 7fc578f89740 1 --2- 172.21.15.44:0/3636643983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7747be40 0x55cc77484f60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc560061d60 tx=0x7fc560059030 comp rx=0 tx=0).stop 2024-04-22T06:32:01.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.956+0000 7fc578f89740 1 -- 172.21.15.44:0/3636643983 shutdown_connections 2024-04-22T06:32:01.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.956+0000 7fc578f89740 1 --2- 172.21.15.44:0/3636643983 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc7743cec0 0x55cc774854a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.956+0000 7fc578f89740 1 --2- 172.21.15.44:0/3636643983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc77438ca0 0x55cc77460ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.956+0000 7fc578f89740 1 --2- 172.21.15.44:0/3636643983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7747be40 0x55cc77484f60 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.956+0000 7fc578f89740 1 -- 172.21.15.44:0/3636643983 >> 172.21.15.44:0/3636643983 conn(0x55cc77437990 msgr2=0x55cc7743a290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:01.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.956+0000 7fc578f89740 1 -- 172.21.15.44:0/3636643983 shutdown_connections 2024-04-22T06:32:01.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.956+0000 7fc578f89740 1 -- 172.21.15.44:0/3636643983 wait complete. 2024-04-22T06:32:01.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.956+0000 7fc578f89740 1 -- start start 2024-04-22T06:32:01.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.957+0000 7fc578f89740 1 Processor -- start 2024-04-22T06:32:01.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.957+0000 7fc578f89740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc7743cec0 0x55cc77483160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.957+0000 7fc578f89740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc7747be40 0x55cc77490780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.957+0000 7fc578f89740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7747fcd0 0x55cc77490cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.957+0000 7fc578f89740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55cc7743b250 con 0x55cc7747fcd0 2024-04-22T06:32:01.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.957+0000 7fc578f89740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55cc77328190 con 0x55cc7743cec0 2024-04-22T06:32:01.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.957+0000 7fc578f89740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55cc7743b530 con 0x55cc7747be40 2024-04-22T06:32:01.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.958+0000 7fc578f89740 10 client.0 _ll_register_callbacks cb 0x55cc773238b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T06:32:01.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.958+0000 7fc574857640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7747fcd0 0x55cc77490cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.958+0000 7fc574857640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7747fcd0 0x55cc77490cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52204/0 (socket says 172.21.15.44:52204) 2024-04-22T06:32:01.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.958+0000 7fc574857640 1 -- 172.21.15.44:0/2612473928 learned_addr learned my addr 172.21.15.44:0/2612473928 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:01.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.958+0000 7fc56f7fe640 1 --2- 172.21.15.44:0/2612473928 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc7747be40 0x55cc77490780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.958+0000 7fc56ffff640 1 --2- 172.21.15.44:0/2612473928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc7743cec0 0x55cc77483160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.958+0000 7fc574857640 1 -- 172.21.15.44:0/2612473928 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc7747be40 msgr2=0x55cc77490780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:01.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.958+0000 7fc574857640 1 --2- 172.21.15.44:0/2612473928 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc7747be40 0x55cc77490780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.958+0000 7fc574857640 1 -- 172.21.15.44:0/2612473928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc7743cec0 msgr2=0x55cc77483160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:01.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.958+0000 7fc574857640 1 --2- 172.21.15.44:0/2612473928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc7743cec0 0x55cc77483160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.958+0000 7fc574857640 1 -- 172.21.15.44:0/2612473928 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc7717c6b0 con 0x55cc7747fcd0 2024-04-22T06:32:01.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.958+0000 7fc574857640 1 --2- 172.21.15.44:0/2612473928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7747fcd0 0x55cc77490cc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc5680076c0 tx=0x7fc568007b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95afc5f9b7208d5a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:01.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.958+0000 7fc54effd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T06:32:01.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.958+0000 7fc54effd640 1 -- 172.21.15.44:0/2612473928 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc568019020 con 0x55cc7747fcd0 2024-04-22T06:32:01.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.958+0000 7fc54effd640 1 -- 172.21.15.44:0/2612473928 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc568012740 con 0x55cc7747fcd0 2024-04-22T06:32:01.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.958+0000 7fc578f89740 1 -- 172.21.15.44:0/2612473928 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc560062290 con 0x55cc7747fcd0 2024-04-22T06:32:01.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.958+0000 7fc54effd640 1 -- 172.21.15.44:0/2612473928 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fc568023050 con 0x55cc7747fcd0 2024-04-22T06:32:01.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.958+0000 7fc578f89740 10 client.4980 fetch_fsmap learned FSMap version 18 2024-04-22T06:32:01.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.958+0000 7fc578f89740 1 -- 172.21.15.44:0/2612473928 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55cc7747eaa0 con 0x55cc7747fcd0 2024-04-22T06:32:01.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.958+0000 7fc54effd640 1 -- 172.21.15.44:0/2612473928 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc560062290 con 0x55cc7747fcd0 2024-04-22T06:32:01.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.959+0000 7fc54effd640 1 -- 172.21.15.44:0/2612473928 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 18) ==== 4792+0+0 (secure 0 0 0) 0x7fc568026020 con 0x55cc7747fcd0 2024-04-22T06:32:01.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.959+0000 7fc578f89740 10 client.4980 fetch_fsmap finished waiting for FSMap version 18 2024-04-22T06:32:01.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.959+0000 7fc578f89740 10 client.4980 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-22T06:32:01.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.959+0000 7fc578f89740 20 client.4980 populate_metadata read hostname 'smithi044' 2024-04-22T06:32:01.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.959+0000 7fc578f89740 1 --2- 172.21.15.44:0/2612473928 >> [v2:172.21.15.70:6834/924955785,v1:172.21.15.70:6838/924955785] conn(0x55cc774360c0 0x55cc7749bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.959+0000 7fc578f89740 4 client.4980 mds_command: new command op to 4305 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T06:32:01.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.959+0000 7fc578f89740 1 -- 172.21.15.44:0/2612473928 --> [v2:172.21.15.70:6834/924955785,v1:172.21.15.70:6838/924955785] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc773ad530 con 0x55cc774360c0 2024-04-22T06:32:01.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.959+0000 7fc56ffff640 1 --2- 172.21.15.44:0/2612473928 >> [v2:172.21.15.70:6834/924955785,v1:172.21.15.70:6838/924955785] conn(0x55cc774360c0 0x55cc7749bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.959+0000 7fc54effd640 10 client.4980 ms_handle_connect on v2:172.21.15.70:6834/924955785 2024-04-22T06:32:01.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.959+0000 7fc56ffff640 1 --2- 172.21.15.44:0/2612473928 >> [v2:172.21.15.70:6834/924955785,v1:172.21.15.70:6838/924955785] conn(0x55cc774360c0 0x55cc7749bf80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6e1b88bc6aefb52d server_cookie=448cbde155da52f3 in_seq=0 out_seq=0 2024-04-22T06:32:01.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.960+0000 7fc54effd640 1 -- 172.21.15.44:0/2612473928 <== mds.1 v2:172.21.15.70:6834/924955785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cc773ad530 con 0x55cc774360c0 2024-04-22T06:32:01.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.960+0000 7fc54effd640 10 client.4980 handle_command_reply: tid=0 2024-04-22T06:32:01.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.960+0000 7fc578f89740 1 client.4980 shutdown 2024-04-22T06:32:01.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.960+0000 7fc578f89740 1 -- 172.21.15.44:0/2612473928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7747fcd0 msgr2=0x55cc77490cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:01.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.960+0000 7fc578f89740 1 --2- 172.21.15.44:0/2612473928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7747fcd0 0x55cc77490cc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc5680076c0 tx=0x7fc568007b80 comp rx=0 tx=0).stop 2024-04-22T06:32:01.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.960+0000 7fc578f89740 1 -- 172.21.15.44:0/2612473928 shutdown_connections 2024-04-22T06:32:01.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.960+0000 7fc578f89740 1 --2- 172.21.15.44:0/2612473928 >> [v2:172.21.15.70:6834/924955785,v1:172.21.15.70:6838/924955785] conn(0x55cc774360c0 0x55cc7749bf80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.960+0000 7fc578f89740 1 --2- 172.21.15.44:0/2612473928 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc7747be40 0x55cc77490780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.960+0000 7fc578f89740 1 --2- 172.21.15.44:0/2612473928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7747fcd0 0x55cc77490cc0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.960+0000 7fc54effd640 0 client.4980 ms_handle_reset on v2:172.21.15.70:6834/924955785 2024-04-22T06:32:01.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.960+0000 7fc578f89740 1 --2- 172.21.15.44:0/2612473928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc7743cec0 0x55cc77483160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.960+0000 7fc578f89740 1 -- 172.21.15.44:0/2612473928 >> 172.21.15.44:0/2612473928 conn(0x55cc77438ca0 msgr2=0x55cc7747c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:01.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.960+0000 7fc578f89740 1 -- 172.21.15.44:0/2612473928 shutdown_connections 2024-04-22T06:32:01.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.960+0000 7fc578f89740 1 -- 172.21.15.44:0/2612473928 wait complete. 2024-04-22T06:32:01.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.960+0000 7fc578f89740 20 client.4980 trim_cache size 0 max 16384 2024-04-22T06:32:01.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.979+0000 7fc578f89740 1 Processor -- start 2024-04-22T06:32:01.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.979+0000 7fc578f89740 1 -- start start 2024-04-22T06:32:01.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.979+0000 7fc578f89740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc7743cec0 0x55cc77490780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.979+0000 7fc578f89740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc7747be40 0x55cc77484e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.979+0000 7fc578f89740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7747fcd0 0x55cc774853c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.979+0000 7fc578f89740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc5701fadf0 con 0x55cc7747fcd0 2024-04-22T06:32:01.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.979+0000 7fc578f89740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55cc77482c40 con 0x55cc7743cec0 2024-04-22T06:32:01.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.979+0000 7fc578f89740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55cc77483950 con 0x55cc7747be40 2024-04-22T06:32:01.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.979+0000 7fc574857640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7747fcd0 0x55cc774853c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.979+0000 7fc574857640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7747fcd0 0x55cc774853c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52214/0 (socket says 172.21.15.44:52214) 2024-04-22T06:32:01.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.979+0000 7fc574857640 1 -- 172.21.15.44:0/1773280493 learned_addr learned my addr 172.21.15.44:0/1773280493 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:01.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.980+0000 7fc56ffff640 1 --2- 172.21.15.44:0/1773280493 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc7743cec0 0x55cc77490780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.980+0000 7fc56f7fe640 1 --2- 172.21.15.44:0/1773280493 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc7747be40 0x55cc77484e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.980+0000 7fc574857640 1 -- 172.21.15.44:0/1773280493 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc7747be40 msgr2=0x55cc77484e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:01.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.980+0000 7fc574857640 1 --2- 172.21.15.44:0/1773280493 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc7747be40 0x55cc77484e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.980+0000 7fc574857640 1 -- 172.21.15.44:0/1773280493 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc7743cec0 msgr2=0x55cc77490780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:01.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.980+0000 7fc574857640 1 --2- 172.21.15.44:0/1773280493 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc7743cec0 0x55cc77490780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.980+0000 7fc574857640 1 -- 172.21.15.44:0/1773280493 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc77438720 con 0x55cc7747fcd0 2024-04-22T06:32:01.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.980+0000 7fc574857640 1 --2- 172.21.15.44:0/1773280493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7747fcd0 0x55cc774853c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc568007300 tx=0x7fc56800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cbdcc2eee5b7ee8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:01.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.980+0000 7fc5557fa640 1 -- 172.21.15.44:0/1773280493 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc568019070 con 0x55cc7747fcd0 2024-04-22T06:32:01.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.980+0000 7fc5557fa640 1 -- 172.21.15.44:0/1773280493 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc56801ee50 con 0x55cc7747fcd0 2024-04-22T06:32:01.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.980+0000 7fc5557fa640 1 -- 172.21.15.44:0/1773280493 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc568012890 con 0x55cc7747fcd0 2024-04-22T06:32:01.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.981+0000 7fc578f89740 1 -- 172.21.15.44:0/1773280493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7747fcd0 msgr2=0x55cc774853c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:01.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.981+0000 7fc578f89740 1 --2- 172.21.15.44:0/1773280493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7747fcd0 0x55cc774853c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc568007300 tx=0x7fc56800f000 comp rx=0 tx=0).stop 2024-04-22T06:32:01.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.981+0000 7fc578f89740 1 -- 172.21.15.44:0/1773280493 shutdown_connections 2024-04-22T06:32:01.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.981+0000 7fc578f89740 1 --2- 172.21.15.44:0/1773280493 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc7747be40 0x55cc77484e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.981+0000 7fc578f89740 1 --2- 172.21.15.44:0/1773280493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7747fcd0 0x55cc774853c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.981+0000 7fc578f89740 1 --2- 172.21.15.44:0/1773280493 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc7743cec0 0x55cc77490780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.981+0000 7fc578f89740 1 -- 172.21.15.44:0/1773280493 >> 172.21.15.44:0/1773280493 conn(0x55cc77438ca0 msgr2=0x55cc77430800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:01.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.981+0000 7fc578f89740 1 -- 172.21.15.44:0/1773280493 shutdown_connections 2024-04-22T06:32:01.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.981+0000 7fc578f89740 1 -- 172.21.15.44:0/1773280493 wait complete. 2024-04-22T06:32:01.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.981+0000 7fc578f89740 1 -- start start 2024-04-22T06:32:01.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.981+0000 7fc578f89740 1 Processor -- start 2024-04-22T06:32:01.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.982+0000 7fc578f89740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc7743cec0 0x55cc7747c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.982+0000 7fc578f89740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc7747be40 0x55cc774f75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.982+0000 7fc578f89740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7747fcd0 0x55cc774a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.982+0000 7fc578f89740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55cc7748d0d0 con 0x55cc7747fcd0 2024-04-22T06:32:01.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.982+0000 7fc578f89740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55cc774364e0 con 0x55cc7743cec0 2024-04-22T06:32:01.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.982+0000 7fc578f89740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55cc7748d740 con 0x55cc7747be40 2024-04-22T06:32:01.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.982+0000 7fc578f89740 10 client.0 _ll_register_callbacks cb 0x7fc568006e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T06:32:01.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.982+0000 7fc574857640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7747fcd0 0x55cc774a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.982+0000 7fc574857640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7747fcd0 0x55cc774a16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52222/0 (socket says 172.21.15.44:52222) 2024-04-22T06:32:01.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.982+0000 7fc56ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc7743cec0 0x55cc7747c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.982+0000 7fc574857640 1 -- 172.21.15.44:0/904071127 learned_addr learned my addr 172.21.15.44:0/904071127 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:01.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.982+0000 7fc56f7fe640 1 --2- 172.21.15.44:0/904071127 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc7747be40 0x55cc774f75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.982+0000 7fc574857640 1 -- 172.21.15.44:0/904071127 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc7747be40 msgr2=0x55cc774f75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:01.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.982+0000 7fc574857640 1 --2- 172.21.15.44:0/904071127 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc7747be40 0x55cc774f75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.983+0000 7fc574857640 1 -- 172.21.15.44:0/904071127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc7743cec0 msgr2=0x55cc7747c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:01.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.983+0000 7fc574857640 1 --2- 172.21.15.44:0/904071127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc7743cec0 0x55cc7747c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.983+0000 7fc574857640 1 -- 172.21.15.44:0/904071127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc774f7b30 con 0x55cc7747fcd0 2024-04-22T06:32:01.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.983+0000 7fc56ffff640 1 --2- 172.21.15.44:0/904071127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc7743cec0 0x55cc7747c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:32:01.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.983+0000 7fc574857640 1 --2- 172.21.15.44:0/904071127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7747fcd0 0x55cc774a16e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc568021040 tx=0x7fc568001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a28177073b1562 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:01.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.983+0000 7fc54effd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T06:32:01.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.983+0000 7fc54effd640 1 -- 172.21.15.44:0/904071127 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc568033070 con 0x55cc7747fcd0 2024-04-22T06:32:01.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.983+0000 7fc54effd640 1 -- 172.21.15.44:0/904071127 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc568012dc0 con 0x55cc7747fcd0 2024-04-22T06:32:01.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.983+0000 7fc578f89740 1 -- 172.21.15.44:0/904071127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc56801ee50 con 0x55cc7747fcd0 2024-04-22T06:32:01.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.983+0000 7fc54effd640 1 -- 172.21.15.44:0/904071127 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fc56803e050 con 0x55cc7747fcd0 2024-04-22T06:32:01.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.983+0000 7fc578f89740 10 client.4986 fetch_fsmap learned FSMap version 18 2024-04-22T06:32:01.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.983+0000 7fc578f89740 1 -- 172.21.15.44:0/904071127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55cc774f16b0 con 0x55cc7747fcd0 2024-04-22T06:32:01.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.983+0000 7fc54effd640 1 -- 172.21.15.44:0/904071127 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc56801ee50 con 0x55cc7747fcd0 2024-04-22T06:32:01.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.983+0000 7fc54effd640 1 -- 172.21.15.44:0/904071127 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 18) ==== 4792+0+0 (secure 0 0 0) 0x7fc568041020 con 0x55cc7747fcd0 2024-04-22T06:32:01.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.983+0000 7fc578f89740 10 client.4986 fetch_fsmap finished waiting for FSMap version 18 2024-04-22T06:32:01.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.983+0000 7fc578f89740 10 client.4986 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-22T06:32:01.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.983+0000 7fc578f89740 20 client.4986 populate_metadata read hostname 'smithi044' 2024-04-22T06:32:01.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.983+0000 7fc578f89740 1 --2- 172.21.15.44:0/904071127 >> [v2:172.21.15.70:6834/924955785,v1:172.21.15.70:6838/924955785] conn(0x55cc77478e30 0x55cc77503a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:01.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.983+0000 7fc578f89740 4 client.4986 mds_command: new command op to 4305 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T06:32:01.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.983+0000 7fc578f89740 1 -- 172.21.15.44:0/904071127 --> [v2:172.21.15.70:6834/924955785,v1:172.21.15.70:6838/924955785] -- command(tid 0: {"prefix": "status"}) -- 0x55cc773a73a0 con 0x55cc77478e30 2024-04-22T06:32:01.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.984+0000 7fc56ffff640 1 --2- 172.21.15.44:0/904071127 >> [v2:172.21.15.70:6834/924955785,v1:172.21.15.70:6838/924955785] conn(0x55cc77478e30 0x55cc77503a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:01.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.984+0000 7fc54effd640 10 client.4986 ms_handle_connect on v2:172.21.15.70:6834/924955785 2024-04-22T06:32:01.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.984+0000 7fc56ffff640 1 --2- 172.21.15.44:0/904071127 >> [v2:172.21.15.70:6834/924955785,v1:172.21.15.70:6838/924955785] conn(0x55cc77478e30 0x55cc77503a10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=732e93bd21122a06 server_cookie=bbf26598de117734 in_seq=0 out_seq=0 2024-04-22T06:32:01.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.984+0000 7fc54effd640 1 -- 172.21.15.44:0/904071127 <== mds.1 v2:172.21.15.70:6834/924955785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55cc773a73a0 con 0x55cc77478e30 2024-04-22T06:32:01.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.984+0000 7fc54effd640 10 client.4986 handle_command_reply: tid=0 2024-04-22T06:32:01.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.984+0000 7fc578f89740 1 client.4986 shutdown 2024-04-22T06:32:01.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.984+0000 7fc578f89740 1 -- 172.21.15.44:0/904071127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7747fcd0 msgr2=0x55cc774a16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:01.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.984+0000 7fc578f89740 1 --2- 172.21.15.44:0/904071127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7747fcd0 0x55cc774a16e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc568021040 tx=0x7fc568001250 comp rx=0 tx=0).stop 2024-04-22T06:32:01.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.984+0000 7fc578f89740 1 -- 172.21.15.44:0/904071127 shutdown_connections 2024-04-22T06:32:01.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.984+0000 7fc578f89740 1 --2- 172.21.15.44:0/904071127 >> [v2:172.21.15.70:6834/924955785,v1:172.21.15.70:6838/924955785] conn(0x55cc77478e30 0x55cc77503a10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.984+0000 7fc578f89740 1 --2- 172.21.15.44:0/904071127 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc7747be40 0x55cc774f75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.984+0000 7fc578f89740 1 --2- 172.21.15.44:0/904071127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7747fcd0 0x55cc774a16e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.984+0000 7fc578f89740 1 --2- 172.21.15.44:0/904071127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc7743cec0 0x55cc7747c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.984+0000 7fc54effd640 0 client.4986 ms_handle_reset on v2:172.21.15.70:6834/924955785 2024-04-22T06:32:01.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.984+0000 7fc578f89740 1 -- 172.21.15.44:0/904071127 >> 172.21.15.44:0/904071127 conn(0x55cc77438ca0 msgr2=0x55cc77484e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:01.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.984+0000 7fc578f89740 1 -- 172.21.15.44:0/904071127 shutdown_connections 2024-04-22T06:32:01.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.984+0000 7fc578f89740 1 -- 172.21.15.44:0/904071127 wait complete. 2024-04-22T06:32:01.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.984+0000 7fc578f89740 20 client.4986 trim_cache size 0 max 16384 2024-04-22T06:32:01.990 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-22T06:32:01.990 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "9b0a34ca-403d-4513-8a66-d3749ba559aa", 2024-04-22T06:32:01.990 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 1, 2024-04-22T06:32:01.990 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 4305, 2024-04-22T06:32:01.990 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-04-22T06:32:01.990 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-04-22T06:32:01.991 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-04-22T06:32:01.991 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 29.904304552999999, 2024-04-22T06:32:01.991 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 18, 2024-04-22T06:32:01.991 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 27, 2024-04-22T06:32:01.991 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 27, 2024-04-22T06:32:01.991 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 61.877729182000003 2024-04-22T06:32:01.991 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-22T06:32:01.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.985+0000 7fc5762ba640 1 -- 172.21.15.44:0/2979005756 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fc55c06c8f0 msgr2=0x7fc55c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:01.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.985+0000 7fc5762ba640 1 --2- 172.21.15.44:0/2979005756 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fc55c06c8f0 0x7fc55c06edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc564036000 tx=0x7fc56403b040 comp rx=0 tx=0).stop 2024-04-22T06:32:01.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.985+0000 7fc5762ba640 1 -- 172.21.15.44:0/2979005756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc57006fa80 msgr2=0x7fc5701ef2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:01.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.985+0000 7fc5762ba640 1 --2- 172.21.15.44:0/2979005756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc57006fa80 0x7fc5701ef2e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc56000e8a0 tx=0x7fc56000ed60 comp rx=0 tx=0).stop 2024-04-22T06:32:01.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.985+0000 7fc5762ba640 1 -- 172.21.15.44:0/2979005756 shutdown_connections 2024-04-22T06:32:01.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.985+0000 7fc5762ba640 1 --2- 172.21.15.44:0/2979005756 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fc55c06c8f0 0x7fc55c06edb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.985+0000 7fc5762ba640 1 --2- 172.21.15.44:0/2979005756 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc57006fe60 0x7fc5701538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.985+0000 7fc5762ba640 1 --2- 172.21.15.44:0/2979005756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc570151c80 0x7fc570152a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.985+0000 7fc5762ba640 1 --2- 172.21.15.44:0/2979005756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc57006fa80 0x7fc5701ef2e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:01.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.985+0000 7fc5762ba640 1 -- 172.21.15.44:0/2979005756 >> 172.21.15.44:0/2979005756 conn(0x7fc5700cd630 msgr2=0x7fc570147f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:01.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.985+0000 7fc5762ba640 1 -- 172.21.15.44:0/2979005756 shutdown_connections 2024-04-22T06:32:01.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:01.985+0000 7fc5762ba640 1 -- 172.21.15.44:0/2979005756 wait complete. 2024-04-22T06:32:01.995 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-22T06:32:02.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.097+0000 7f8f204c1640 1 Processor -- start 2024-04-22T06:32:02.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.097+0000 7f8f204c1640 1 -- start start 2024-04-22T06:32:02.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.098+0000 7f8f204c1640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f1815f050 0x7f8f18073230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:02.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.098+0000 7f8f204c1640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f8f18073770 0x7f8f180c5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:02.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.098+0000 7f8f204c1640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f180ccde0 con 0x7f8f18155460 2024-04-22T06:32:02.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.098+0000 7f8f204c1640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f8f18155840 con 0x7f8f18073770 2024-04-22T06:32:02.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.098+0000 7f8f204c1640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8f180d1520 con 0x7f8f1815f050 2024-04-22T06:32:02.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.099+0000 7f8f1ea5e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f8f18073770 0x7f8f180c5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:02.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.099+0000 7f8f1ea5e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f8f18073770 0x7f8f180c5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52234/0 (socket says 172.21.15.44:52234) 2024-04-22T06:32:02.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.099+0000 7f8f1ea5e640 1 -- 172.21.15.44:0/2267146377 learned_addr learned my addr 172.21.15.44:0/2267146377 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:02.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.099+0000 7f8f1da5c640 1 --2- 172.21.15.44:0/2267146377 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f1815f050 0x7f8f18073230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:02.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.099+0000 7f8f1ea5e640 1 -- 172.21.15.44:0/2267146377 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f1815f050 msgr2=0x7f8f18073230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:02.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.099+0000 7f8f1ea5e640 1 --2- 172.21.15.44:0/2267146377 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f1815f050 0x7f8f18073230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.099+0000 7f8f1ea5e640 1 -- 172.21.15.44:0/2267146377 >> v1:172.21.15.70:6789/0 conn(0x7f8f18155460 legacy=0x7f8f180cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:02.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.099+0000 7f8f1ea5e640 1 -- 172.21.15.44:0/2267146377 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f180cd0a0 con 0x7f8f18073770 2024-04-22T06:32:02.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.099+0000 7f8f1da5c640 1 --2- 172.21.15.44:0/2267146377 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f1815f050 0x7f8f18073230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:32:02.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.100+0000 7f8f1ea5e640 1 --2- 172.21.15.44:0/2267146377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f18073770 0x7f8f180c5c00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8f10009e50 tx=0x7f8f1002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5e95d100017e501 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:02.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.100+0000 7f8f1d25b640 1 -- 172.21.15.44:0/2267146377 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1002ca70 con 0x7f8f18073770 2024-04-22T06:32:02.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.100+0000 7f8f1d25b640 1 -- 172.21.15.44:0/2267146377 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f1002cc10 con 0x7f8f18073770 2024-04-22T06:32:02.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.100+0000 7f8f1d25b640 1 -- 172.21.15.44:0/2267146377 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f100343f0 con 0x7f8f18073770 2024-04-22T06:32:02.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.100+0000 7f8f204c1640 1 -- 172.21.15.44:0/2267146377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f18073770 msgr2=0x7f8f180c5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:02.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.100+0000 7f8f204c1640 1 --2- 172.21.15.44:0/2267146377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f18073770 0x7f8f180c5c00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8f10009e50 tx=0x7f8f1002bed0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.100+0000 7f8f204c1640 1 -- 172.21.15.44:0/2267146377 shutdown_connections 2024-04-22T06:32:02.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.100+0000 7f8f204c1640 1 --2- 172.21.15.44:0/2267146377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f18073770 0x7f8f180c5c00 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.100+0000 7f8f204c1640 1 --2- 172.21.15.44:0/2267146377 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f1815f050 0x7f8f18073230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.100+0000 7f8f204c1640 1 -- 172.21.15.44:0/2267146377 >> 172.21.15.44:0/2267146377 conn(0x7f8f1806cca0 msgr2=0x7f8f1806d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:02.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.101+0000 7f8f204c1640 1 -- 172.21.15.44:0/2267146377 shutdown_connections 2024-04-22T06:32:02.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.101+0000 7f8f204c1640 1 -- 172.21.15.44:0/2267146377 wait complete. 2024-04-22T06:32:02.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.102+0000 7f8f204c1640 1 Processor -- start 2024-04-22T06:32:02.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.102+0000 7f8f204c1640 1 -- start start 2024-04-22T06:32:02.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.102+0000 7f8f204c1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f18073770 0x7f8f1814b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:02.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.102+0000 7f8f204c1640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f18155460 0x7f8f1814deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:02.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.102+0000 7f8f204c1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f1815f050 0x7f8f1814e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:02.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.102+0000 7f8f204c1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8f180d0ed0 con 0x7f8f1815f050 2024-04-22T06:32:02.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.102+0000 7f8f204c1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8f180d0bf0 con 0x7f8f18073770 2024-04-22T06:32:02.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.102+0000 7f8f204c1640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8f180d11b0 con 0x7f8f18155460 2024-04-22T06:32:02.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.102+0000 7f8f1ea5e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f1815f050 0x7f8f1814e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:02.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.103+0000 7f8f1ea5e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f1815f050 0x7f8f1814e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52244/0 (socket says 172.21.15.44:52244) 2024-04-22T06:32:02.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.103+0000 7f8f1ea5e640 1 -- 172.21.15.44:0/263615081 learned_addr learned my addr 172.21.15.44:0/263615081 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:02.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.103+0000 7f8f1da5c640 1 --2- 172.21.15.44:0/263615081 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f18155460 0x7f8f1814deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:02.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.103+0000 7f8f1e25d640 1 --2- 172.21.15.44:0/263615081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f18073770 0x7f8f1814b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:02.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.103+0000 7f8f1ea5e640 1 -- 172.21.15.44:0/263615081 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f18155460 msgr2=0x7f8f1814deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:02.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.103+0000 7f8f1ea5e640 1 --2- 172.21.15.44:0/263615081 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f18155460 0x7f8f1814deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.103+0000 7f8f1ea5e640 1 -- 172.21.15.44:0/263615081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f18073770 msgr2=0x7f8f1814b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:02.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.103+0000 7f8f1ea5e640 1 --2- 172.21.15.44:0/263615081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f18073770 0x7f8f1814b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.103+0000 7f8f1ea5e640 1 -- 172.21.15.44:0/263615081 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f1814c040 con 0x7f8f1815f050 2024-04-22T06:32:02.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.103+0000 7f8f1ea5e640 1 --2- 172.21.15.44:0/263615081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f1815f050 0x7f8f1814e3f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8f10009e50 tx=0x7f8f10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:02.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.103+0000 7f8f0effd640 1 -- 172.21.15.44:0/263615081 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f10043020 con 0x7f8f1815f050 2024-04-22T06:32:02.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.103+0000 7f8f0effd640 1 -- 172.21.15.44:0/263615081 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f1003e450 con 0x7f8f1815f050 2024-04-22T06:32:02.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.103+0000 7f8f0effd640 1 -- 172.21.15.44:0/263615081 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1003e820 con 0x7f8f1815f050 2024-04-22T06:32:02.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.103+0000 7f8f204c1640 1 -- 172.21.15.44:0/263615081 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f180caad0 con 0x7f8f1815f050 2024-04-22T06:32:02.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.103+0000 7f8f204c1640 1 -- 172.21.15.44:0/263615081 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f1814c5e0 con 0x7f8f1815f050 2024-04-22T06:32:02.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.105+0000 7f8f0effd640 1 -- 172.21.15.44:0/263615081 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8f10046020 con 0x7f8f1815f050 2024-04-22T06:32:02.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.105+0000 7f8f0effd640 1 --2- 172.21.15.44:0/263615081 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f8efc06c8f0 0x7f8efc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:02.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.105+0000 7f8f0effd640 1 -- 172.21.15.44:0/263615081 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f8f100ae840 con 0x7f8f1815f050 2024-04-22T06:32:02.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.105+0000 7f8f1e25d640 1 --2- 172.21.15.44:0/263615081 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f8efc06c8f0 0x7f8efc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:02.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.106+0000 7f8f1e25d640 1 --2- 172.21.15.44:0/263615081 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f8efc06c8f0 0x7f8efc06edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8f14004ea0 tx=0x7f8f14009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:02.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.106+0000 7f8f23192740 1 Processor -- start 2024-04-22T06:32:02.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.106+0000 7f8f23192740 1 -- start start 2024-04-22T06:32:02.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.107+0000 7f8f23192740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562f05403e40 0x562f0540cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:02.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.107+0000 7f8f23192740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562f053c4ec0 0x562f0540d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:02.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.107+0000 7f8f23192740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562f053c0ca0 0x562f053e8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:02.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.107+0000 7f8f23192740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562f05327b10 con 0x562f053c4ec0 2024-04-22T06:32:02.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.107+0000 7f8f23192740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x562f052c5240 con 0x562f05403e40 2024-04-22T06:32:02.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.107+0000 7f8f23192740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x562f05323ad0 con 0x562f053c0ca0 2024-04-22T06:32:02.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.107+0000 7f8f1da5c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562f053c4ec0 0x562f0540d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:02.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.107+0000 7f8f1ea5e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562f053c0ca0 0x562f053e8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:02.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.107+0000 7f8f1da5c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562f053c4ec0 0x562f0540d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52246/0 (socket says 172.21.15.44:52246) 2024-04-22T06:32:02.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.107+0000 7f8f1e25d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562f05403e40 0x562f0540cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:02.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.107+0000 7f8f1da5c640 1 -- 172.21.15.44:0/1003582974 learned_addr learned my addr 172.21.15.44:0/1003582974 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:02.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.107+0000 7f8f1ea5e640 1 -- 172.21.15.44:0/1003582974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562f05403e40 msgr2=0x562f0540cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:02.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.107+0000 7f8f1ea5e640 1 --2- 172.21.15.44:0/1003582974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562f05403e40 0x562f0540cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.107+0000 7f8f1ea5e640 1 -- 172.21.15.44:0/1003582974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562f053c4ec0 msgr2=0x562f0540d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:02.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.107+0000 7f8f1ea5e640 1 --2- 172.21.15.44:0/1003582974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562f053c4ec0 0x562f0540d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.107+0000 7f8f1ea5e640 1 -- 172.21.15.44:0/1003582974 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f0503a2d0 con 0x562f053c0ca0 2024-04-22T06:32:02.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.107+0000 7f8f1da5c640 1 --2- 172.21.15.44:0/1003582974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562f053c4ec0 0x562f0540d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:32:02.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.108+0000 7f8f1ea5e640 1 --2- 172.21.15.44:0/1003582974 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562f053c0ca0 0x562f053e8ee0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8f100ac940 tx=0x7f8f1007bd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d7bc7a7bcf4c996 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:02.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.108+0000 7f8efbfff640 1 -- 172.21.15.44:0/1003582974 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f10080520 con 0x562f053c0ca0 2024-04-22T06:32:02.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.108+0000 7f8efbfff640 1 -- 172.21.15.44:0/1003582974 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f10084d50 con 0x562f053c0ca0 2024-04-22T06:32:02.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.108+0000 7f8efbfff640 1 -- 172.21.15.44:0/1003582974 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f100787e0 con 0x562f053c0ca0 2024-04-22T06:32:02.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.108+0000 7f8f23192740 1 -- 172.21.15.44:0/1003582974 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562f053c0ca0 msgr2=0x562f053e8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:02.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.108+0000 7f8f23192740 1 --2- 172.21.15.44:0/1003582974 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562f053c0ca0 0x562f053e8ee0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8f100ac940 tx=0x7f8f1007bd70 comp rx=0 tx=0).stop 2024-04-22T06:32:02.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.109+0000 7f8f23192740 1 -- 172.21.15.44:0/1003582974 shutdown_connections 2024-04-22T06:32:02.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.109+0000 7f8f23192740 1 --2- 172.21.15.44:0/1003582974 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562f053c0ca0 0x562f053e8ee0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.109+0000 7f8f23192740 1 --2- 172.21.15.44:0/1003582974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562f053c4ec0 0x562f0540d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.109+0000 7f8f23192740 1 --2- 172.21.15.44:0/1003582974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562f05403e40 0x562f0540cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.109+0000 7f8f23192740 1 -- 172.21.15.44:0/1003582974 >> 172.21.15.44:0/1003582974 conn(0x562f053bf990 msgr2=0x562f053c2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:02.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.109+0000 7f8f23192740 1 -- 172.21.15.44:0/1003582974 shutdown_connections 2024-04-22T06:32:02.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.109+0000 7f8f23192740 1 -- 172.21.15.44:0/1003582974 wait complete. 2024-04-22T06:32:02.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.109+0000 7f8f23192740 1 -- start start 2024-04-22T06:32:02.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.109+0000 7f8f23192740 1 Processor -- start 2024-04-22T06:32:02.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.110+0000 7f8f23192740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562f053c4ec0 0x562f054187a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:02.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.110+0000 7f8f23192740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562f05403e40 0x562f05409610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:02.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.110+0000 7f8f23192740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562f053fd880 0x562f053fdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:02.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.110+0000 7f8f23192740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562f05323ad0 con 0x562f05403e40 2024-04-22T06:32:02.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.110+0000 7f8f23192740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x562f052b0190 con 0x562f053c4ec0 2024-04-22T06:32:02.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.110+0000 7f8f23192740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x562f053c3250 con 0x562f053fd880 2024-04-22T06:32:02.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.110+0000 7f8f23192740 10 client.0 _ll_register_callbacks cb 0x562f052ab8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T06:32:02.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.110+0000 7f8f1da5c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562f05403e40 0x562f05409610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:02.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.110+0000 7f8f1da5c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562f05403e40 0x562f05409610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52262/0 (socket says 172.21.15.44:52262) 2024-04-22T06:32:02.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.110+0000 7f8f1da5c640 1 -- 172.21.15.44:0/181762646 learned_addr learned my addr 172.21.15.44:0/181762646 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:02.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.110+0000 7f8f1ea5e640 1 --2- 172.21.15.44:0/181762646 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562f053fd880 0x562f053fdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:02.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.110+0000 7f8f1e25d640 1 --2- 172.21.15.44:0/181762646 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562f053c4ec0 0x562f054187a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:02.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.111+0000 7f8f1da5c640 1 -- 172.21.15.44:0/181762646 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562f053fd880 msgr2=0x562f053fdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:02.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.111+0000 7f8f1da5c640 1 --2- 172.21.15.44:0/181762646 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562f053fd880 0x562f053fdd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.111+0000 7f8f1da5c640 1 -- 172.21.15.44:0/181762646 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562f053c4ec0 msgr2=0x562f054187a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:02.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.111+0000 7f8f1da5c640 1 --2- 172.21.15.44:0/181762646 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562f053c4ec0 0x562f054187a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.111+0000 7f8f1da5c640 1 -- 172.21.15.44:0/181762646 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f051046b0 con 0x562f05403e40 2024-04-22T06:32:02.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.111+0000 7f8f1ea5e640 1 --2- 172.21.15.44:0/181762646 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562f053fd880 0x562f053fdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:32:02.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.111+0000 7f8f1da5c640 1 --2- 172.21.15.44:0/181762646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562f05403e40 0x562f05409610 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8f08003d00 tx=0x7f8f0800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e657547f45c194 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:02.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.111+0000 7f8ef97fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T06:32:02.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.111+0000 7f8ef97fa640 1 -- 172.21.15.44:0/181762646 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08018070 con 0x562f05403e40 2024-04-22T06:32:02.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.111+0000 7f8ef97fa640 1 -- 172.21.15.44:0/181762646 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f8f08014650 con 0x562f05403e40 2024-04-22T06:32:02.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.111+0000 7f8f23192740 1 -- 172.21.15.44:0/181762646 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f10084d50 con 0x562f05403e40 2024-04-22T06:32:02.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.111+0000 7f8ef97fa640 1 -- 172.21.15.44:0/181762646 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f8f08023050 con 0x562f05403e40 2024-04-22T06:32:02.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.111+0000 7f8f23192740 10 client.4998 fetch_fsmap learned FSMap version 18 2024-04-22T06:32:02.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.111+0000 7f8f23192740 1 -- 172.21.15.44:0/181762646 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x562f0540c5a0 con 0x562f05403e40 2024-04-22T06:32:02.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.111+0000 7f8ef97fa640 1 -- 172.21.15.44:0/181762646 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f10084d50 con 0x562f05403e40 2024-04-22T06:32:02.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.112+0000 7f8ef97fa640 1 -- 172.21.15.44:0/181762646 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 18) ==== 4792+0+0 (secure 0 0 0) 0x7f8f08026020 con 0x562f05403e40 2024-04-22T06:32:02.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.112+0000 7f8f23192740 10 client.4998 fetch_fsmap finished waiting for FSMap version 18 2024-04-22T06:32:02.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.112+0000 7f8f23192740 10 client.4998 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-22T06:32:02.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.112+0000 7f8f23192740 20 client.4998 populate_metadata read hostname 'smithi044' 2024-04-22T06:32:02.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.112+0000 7f8f23192740 1 --2- 172.21.15.44:0/181762646 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x562f054218a0 0x562f05423c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:02.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.112+0000 7f8f23192740 4 client.4998 mds_command: new command op to 4308 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T06:32:02.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.112+0000 7f8f23192740 1 -- 172.21.15.44:0/181762646 --> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f05335530 con 0x562f054218a0 2024-04-22T06:32:02.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.112+0000 7f8f1e25d640 1 --2- 172.21.15.44:0/181762646 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x562f054218a0 0x562f05423c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:02.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.112+0000 7f8f1e25d640 1 --2- 172.21.15.44:0/181762646 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x562f054218a0 0x562f05423c90 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=489f123fe4535797 server_cookie=d61ca16a0ae352c1 in_seq=0 out_seq=0 2024-04-22T06:32:02.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.112+0000 7f8ef97fa640 10 client.4998 ms_handle_connect on v2:172.21.15.70:6837/2915150040 2024-04-22T06:32:02.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.113+0000 7f8ef97fa640 1 -- 172.21.15.44:0/181762646 <== mds.0 v2:172.21.15.70:6837/2915150040 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562f05335530 con 0x562f054218a0 2024-04-22T06:32:02.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.113+0000 7f8ef97fa640 10 client.4998 handle_command_reply: tid=0 2024-04-22T06:32:02.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.113+0000 7f8f23192740 1 client.4998 shutdown 2024-04-22T06:32:02.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.113+0000 7f8f23192740 1 -- 172.21.15.44:0/181762646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562f05403e40 msgr2=0x562f05409610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:02.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.113+0000 7f8f23192740 1 --2- 172.21.15.44:0/181762646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562f05403e40 0x562f05409610 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8f08003d00 tx=0x7f8f0800c910 comp rx=0 tx=0).stop 2024-04-22T06:32:02.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.113+0000 7f8f23192740 1 -- 172.21.15.44:0/181762646 shutdown_connections 2024-04-22T06:32:02.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.113+0000 7f8f23192740 1 --2- 172.21.15.44:0/181762646 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562f053fd880 0x562f053fdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.113+0000 7f8f23192740 1 --2- 172.21.15.44:0/181762646 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x562f054218a0 0x562f05423c90 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.113+0000 7f8f23192740 1 --2- 172.21.15.44:0/181762646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562f05403e40 0x562f05409610 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.113+0000 7f8ef97fa640 0 client.4998 ms_handle_reset on v2:172.21.15.70:6837/2915150040 2024-04-22T06:32:02.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.113+0000 7f8f23192740 1 --2- 172.21.15.44:0/181762646 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562f053c4ec0 0x562f054187a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.113+0000 7f8f23192740 1 -- 172.21.15.44:0/181762646 >> 172.21.15.44:0/181762646 conn(0x562f053c0ca0 msgr2=0x562f05403440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:02.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.113+0000 7f8f23192740 1 -- 172.21.15.44:0/181762646 shutdown_connections 2024-04-22T06:32:02.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.113+0000 7f8f23192740 1 -- 172.21.15.44:0/181762646 wait complete. 2024-04-22T06:32:02.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.113+0000 7f8f23192740 20 client.4998 trim_cache size 0 max 16384 2024-04-22T06:32:02.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.131+0000 7f8f23192740 1 Processor -- start 2024-04-22T06:32:02.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.131+0000 7f8f23192740 1 -- start start 2024-04-22T06:32:02.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.132+0000 7f8f23192740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562f053c4ec0 0x562f054084d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:02.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.132+0000 7f8f23192740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562f053fd880 0x562f05408a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:02.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.132+0000 7f8f23192740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562f05403e40 0x562f05428cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:02.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.132+0000 7f8f23192740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8f181f50a0 con 0x562f053fd880 2024-04-22T06:32:02.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.132+0000 7f8f23192740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x562f053bc170 con 0x562f05403e40 2024-04-22T06:32:02.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.132+0000 7f8f23192740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x562f05417cc0 con 0x562f053c4ec0 2024-04-22T06:32:02.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.132+0000 7f8f1da5c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562f053fd880 0x562f05408a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:02.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.132+0000 7f8f1da5c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562f053fd880 0x562f05408a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52276/0 (socket says 172.21.15.44:52276) 2024-04-22T06:32:02.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.132+0000 7f8f1da5c640 1 -- 172.21.15.44:0/247770777 learned_addr learned my addr 172.21.15.44:0/247770777 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:02.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.132+0000 7f8f1e25d640 1 --2- 172.21.15.44:0/247770777 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562f053c4ec0 0x562f054084d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:02.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.132+0000 7f8f1ea5e640 1 --2- 172.21.15.44:0/247770777 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562f05403e40 0x562f05428cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:02.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.132+0000 7f8f1da5c640 1 -- 172.21.15.44:0/247770777 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562f053c4ec0 msgr2=0x562f054084d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:02.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.132+0000 7f8f1da5c640 1 --2- 172.21.15.44:0/247770777 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562f053c4ec0 0x562f054084d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.132+0000 7f8f1da5c640 1 -- 172.21.15.44:0/247770777 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562f05403e40 msgr2=0x562f05428cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:02.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.132+0000 7f8f1da5c640 1 --2- 172.21.15.44:0/247770777 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562f05403e40 0x562f05428cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.133+0000 7f8f1da5c640 1 -- 172.21.15.44:0/247770777 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f053d9340 con 0x562f053fd880 2024-04-22T06:32:02.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.133+0000 7f8f1ea5e640 1 --2- 172.21.15.44:0/247770777 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562f05403e40 0x562f05428cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:32:02.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.133+0000 7f8f1da5c640 1 --2- 172.21.15.44:0/247770777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562f053fd880 0x562f05408a10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8f0800c3a0 tx=0x7f8f0800e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb2b34f25698c67d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:02.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.133+0000 7f8efb7fe640 1 -- 172.21.15.44:0/247770777 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08018040 con 0x562f053fd880 2024-04-22T06:32:02.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.133+0000 7f8efb7fe640 1 -- 172.21.15.44:0/247770777 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f080148f0 con 0x562f053fd880 2024-04-22T06:32:02.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.133+0000 7f8efb7fe640 1 -- 172.21.15.44:0/247770777 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08014e50 con 0x562f053fd880 2024-04-22T06:32:02.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.133+0000 7f8f23192740 1 -- 172.21.15.44:0/247770777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562f053fd880 msgr2=0x562f05408a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:02.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.133+0000 7f8f23192740 1 --2- 172.21.15.44:0/247770777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562f053fd880 0x562f05408a10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8f0800c3a0 tx=0x7f8f0800e000 comp rx=0 tx=0).stop 2024-04-22T06:32:02.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.133+0000 7f8f23192740 1 -- 172.21.15.44:0/247770777 shutdown_connections 2024-04-22T06:32:02.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.133+0000 7f8f23192740 1 --2- 172.21.15.44:0/247770777 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562f05403e40 0x562f05428cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.133+0000 7f8f23192740 1 --2- 172.21.15.44:0/247770777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562f053fd880 0x562f05408a10 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.134+0000 7f8f23192740 1 --2- 172.21.15.44:0/247770777 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562f053c4ec0 0x562f054084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.134+0000 7f8f23192740 1 -- 172.21.15.44:0/247770777 >> 172.21.15.44:0/247770777 conn(0x562f053c0ca0 msgr2=0x562f053b8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:02.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.134+0000 7f8f23192740 1 -- 172.21.15.44:0/247770777 shutdown_connections 2024-04-22T06:32:02.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.134+0000 7f8f23192740 1 -- 172.21.15.44:0/247770777 wait complete. 2024-04-22T06:32:02.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.134+0000 7f8f23192740 1 -- start start 2024-04-22T06:32:02.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.134+0000 7f8f23192740 1 Processor -- start 2024-04-22T06:32:02.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.135+0000 7f8f23192740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562f053c4ec0 0x562f054822d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:02.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.135+0000 7f8f23192740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562f053fd880 0x562f05408680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:02.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.135+0000 7f8f23192740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562f05403e40 0x562f05478b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:02.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.135+0000 7f8f23192740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562f053e58b0 con 0x562f053c4ec0 2024-04-22T06:32:02.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.135+0000 7f8f23192740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x562f0540a2d0 con 0x562f053fd880 2024-04-22T06:32:02.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.135+0000 7f8f23192740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x562f0540b1b0 con 0x562f05403e40 2024-04-22T06:32:02.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.135+0000 7f8f23192740 10 client.0 _ll_register_callbacks cb 0x7f8f08004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T06:32:02.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.135+0000 7f8f1e25d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562f053c4ec0 0x562f054822d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:02.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.135+0000 7f8f1e25d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562f053c4ec0 0x562f054822d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52288/0 (socket says 172.21.15.44:52288) 2024-04-22T06:32:02.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.135+0000 7f8f1e25d640 1 -- 172.21.15.44:0/2163744655 learned_addr learned my addr 172.21.15.44:0/2163744655 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:02.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.135+0000 7f8f1da5c640 1 --2- 172.21.15.44:0/2163744655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562f053fd880 0x562f05408680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:02.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.135+0000 7f8f1ea5e640 1 --2- 172.21.15.44:0/2163744655 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562f05403e40 0x562f05478b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:02.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.135+0000 7f8f1e25d640 1 -- 172.21.15.44:0/2163744655 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562f05403e40 msgr2=0x562f05478b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:02.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.135+0000 7f8f1e25d640 1 --2- 172.21.15.44:0/2163744655 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562f05403e40 0x562f05478b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.135+0000 7f8f1e25d640 1 -- 172.21.15.44:0/2163744655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562f053fd880 msgr2=0x562f05408680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:02.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.135+0000 7f8f1e25d640 1 --2- 172.21.15.44:0/2163744655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562f053fd880 0x562f05408680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.135+0000 7f8f1e25d640 1 -- 172.21.15.44:0/2163744655 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f053c2b20 con 0x562f053c4ec0 2024-04-22T06:32:02.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.136+0000 7f8f1e25d640 1 --2- 172.21.15.44:0/2163744655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562f053c4ec0 0x562f054822d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x562f05409f60 tx=0x7f8f14011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27452722bd04a30d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:02.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.136+0000 7f8ef97fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T06:32:02.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.136+0000 7f8ef97fa640 1 -- 172.21.15.44:0/2163744655 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1402ad60 con 0x562f053c4ec0 2024-04-22T06:32:02.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.136+0000 7f8ef97fa640 1 -- 172.21.15.44:0/2163744655 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f8f14010040 con 0x562f053c4ec0 2024-04-22T06:32:02.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.136+0000 7f8f23192740 1 -- 172.21.15.44:0/2163744655 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f080148f0 con 0x562f053c4ec0 2024-04-22T06:32:02.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.136+0000 7f8ef97fa640 1 -- 172.21.15.44:0/2163744655 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f8f14039050 con 0x562f053c4ec0 2024-04-22T06:32:02.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.136+0000 7f8f23192740 10 client.5004 fetch_fsmap learned FSMap version 18 2024-04-22T06:32:02.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.136+0000 7f8f23192740 1 -- 172.21.15.44:0/2163744655 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x562f053e5fd0 con 0x562f053c4ec0 2024-04-22T06:32:02.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.136+0000 7f8ef97fa640 1 -- 172.21.15.44:0/2163744655 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f080148f0 con 0x562f053c4ec0 2024-04-22T06:32:02.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.136+0000 7f8ef97fa640 1 -- 172.21.15.44:0/2163744655 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 18) ==== 4792+0+0 (secure 0 0 0) 0x7f8f1403c020 con 0x562f053c4ec0 2024-04-22T06:32:02.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.136+0000 7f8f23192740 10 client.5004 fetch_fsmap finished waiting for FSMap version 18 2024-04-22T06:32:02.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.136+0000 7f8f23192740 10 client.5004 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-22T06:32:02.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.136+0000 7f8f23192740 20 client.5004 populate_metadata read hostname 'smithi044' 2024-04-22T06:32:02.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.136+0000 7f8f23192740 1 --2- 172.21.15.44:0/2163744655 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x562f054079b0 0x562f0548b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:02.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.136+0000 7f8f23192740 4 client.5004 mds_command: new command op to 4308 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T06:32:02.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.136+0000 7f8f23192740 1 -- 172.21.15.44:0/2163744655 --> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] -- command(tid 0: {"prefix": "status"}) -- 0x562f0532f3a0 con 0x562f054079b0 2024-04-22T06:32:02.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.137+0000 7f8f1da5c640 1 --2- 172.21.15.44:0/2163744655 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x562f054079b0 0x562f0548b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:02.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.137+0000 7f8f1da5c640 1 --2- 172.21.15.44:0/2163744655 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x562f054079b0 0x562f0548b670 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8125793781dec08f server_cookie=22dd223abc707b78 in_seq=0 out_seq=0 2024-04-22T06:32:02.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.137+0000 7f8ef97fa640 10 client.5004 ms_handle_connect on v2:172.21.15.70:6837/2915150040 2024-04-22T06:32:02.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.137+0000 7f8ef97fa640 1 -- 172.21.15.44:0/2163744655 <== mds.0 v2:172.21.15.70:6837/2915150040 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x562f0532f3a0 con 0x562f054079b0 2024-04-22T06:32:02.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.137+0000 7f8ef97fa640 10 client.5004 handle_command_reply: tid=0 2024-04-22T06:32:02.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.137+0000 7f8f23192740 1 client.5004 shutdown 2024-04-22T06:32:02.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.137+0000 7f8f23192740 1 -- 172.21.15.44:0/2163744655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562f053c4ec0 msgr2=0x562f054822d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:02.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.137+0000 7f8f23192740 1 --2- 172.21.15.44:0/2163744655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562f053c4ec0 0x562f054822d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x562f05409f60 tx=0x7f8f14011000 comp rx=0 tx=0).stop 2024-04-22T06:32:02.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.137+0000 7f8f23192740 1 -- 172.21.15.44:0/2163744655 shutdown_connections 2024-04-22T06:32:02.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.137+0000 7f8f23192740 1 --2- 172.21.15.44:0/2163744655 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562f05403e40 0x562f05478b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.137+0000 7f8f23192740 1 --2- 172.21.15.44:0/2163744655 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x562f054079b0 0x562f0548b670 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.137+0000 7f8f23192740 1 --2- 172.21.15.44:0/2163744655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562f053fd880 0x562f05408680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.137+0000 7f8ef97fa640 0 client.5004 ms_handle_reset on v2:172.21.15.70:6837/2915150040 2024-04-22T06:32:02.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.137+0000 7f8f23192740 1 --2- 172.21.15.44:0/2163744655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562f053c4ec0 0x562f054822d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.137+0000 7f8f23192740 1 -- 172.21.15.44:0/2163744655 >> 172.21.15.44:0/2163744655 conn(0x562f053c0ca0 msgr2=0x562f05418520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:02.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.137+0000 7f8f23192740 1 -- 172.21.15.44:0/2163744655 shutdown_connections 2024-04-22T06:32:02.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.137+0000 7f8f23192740 1 -- 172.21.15.44:0/2163744655 wait complete. 2024-04-22T06:32:02.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.138+0000 7f8f23192740 20 client.5004 trim_cache size 0 max 16384 2024-04-22T06:32:02.143 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-22T06:32:02.143 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "9b0a34ca-403d-4513-8a66-d3749ba559aa", 2024-04-22T06:32:02.143 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-04-22T06:32:02.143 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 4308, 2024-04-22T06:32:02.143 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-04-22T06:32:02.143 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-04-22T06:32:02.143 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-04-22T06:32:02.143 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 55.129942460999999, 2024-04-22T06:32:02.143 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 18, 2024-04-22T06:32:02.143 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 27, 2024-04-22T06:32:02.144 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 27, 2024-04-22T06:32:02.144 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 62.029862706000003 2024-04-22T06:32:02.144 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-22T06:32:02.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.138+0000 7f8f204c1640 1 -- 172.21.15.44:0/263615081 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f8efc06c8f0 msgr2=0x7f8efc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:02.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.138+0000 7f8f204c1640 1 --2- 172.21.15.44:0/263615081 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f8efc06c8f0 0x7f8efc06edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8f14004ea0 tx=0x7f8f14009040 comp rx=0 tx=0).stop 2024-04-22T06:32:02.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.138+0000 7f8f204c1640 1 -- 172.21.15.44:0/263615081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f1815f050 msgr2=0x7f8f1814e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:02.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.138+0000 7f8f204c1640 1 --2- 172.21.15.44:0/263615081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f1815f050 0x7f8f1814e3f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8f10009e50 tx=0x7f8f10004a40 comp rx=0 tx=0).stop 2024-04-22T06:32:02.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.138+0000 7f8f204c1640 1 -- 172.21.15.44:0/263615081 shutdown_connections 2024-04-22T06:32:02.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.138+0000 7f8f204c1640 1 --2- 172.21.15.44:0/263615081 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f8efc06c8f0 0x7f8efc06edb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.138+0000 7f8f204c1640 1 --2- 172.21.15.44:0/263615081 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f18155460 0x7f8f1814deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.138+0000 7f8f204c1640 1 --2- 172.21.15.44:0/263615081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f1815f050 0x7f8f1814e3f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.138+0000 7f8f204c1640 1 --2- 172.21.15.44:0/263615081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f18073770 0x7f8f1814b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.138+0000 7f8f204c1640 1 -- 172.21.15.44:0/263615081 >> 172.21.15.44:0/263615081 conn(0x7f8f1806cca0 msgr2=0x7f8f18153f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:02.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.138+0000 7f8f204c1640 1 -- 172.21.15.44:0/263615081 shutdown_connections 2024-04-22T06:32:02.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.138+0000 7f8f204c1640 1 -- 172.21.15.44:0/263615081 wait complete. 2024-04-22T06:32:02.148 INFO:teuthology.run_tasks:Running task sequential... 2024-04-22T06:32:02.158 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-22T06:32:02.158 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-22T06:32:02.158 INFO:teuthology.task.print:Enabling mgr modules 2024-04-22T06:32:02.159 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-22T06:32:02.219 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-22T06:32:02.219 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-04-22T06:32:02.219 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-22T06:32:02.219 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-22T06:32:02.219 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-22T06:32:02.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T06:32:02.219 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-04-22T06:32:02.235 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-22T06:32:02.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T06:32:02.236 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-22T06:32:02.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T06:32:02.300 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link delete ceph-brx 2024-04-22T06:32:02.404 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-22T06:32:02.405 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-22T06:32:02.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.508+0000 7f7473225640 1 Processor -- start 2024-04-22T06:32:02.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.508+0000 7f7473225640 1 -- start start 2024-04-22T06:32:02.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.509+0000 7f7473225640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f746c151f50 0x7f746c158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:02.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.509+0000 7f7473225640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f746c1598a0 0x7f746c159d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:02.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.509+0000 7f7473225640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f746c15a270 con 0x7f746c0c8410 2024-04-22T06:32:02.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.509+0000 7f7473225640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f746c158db0 con 0x7f746c1598a0 2024-04-22T06:32:02.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.509+0000 7f7473225640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f746c15b770 con 0x7f746c151f50 2024-04-22T06:32:02.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.509+0000 7f746bfff640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f746c0c8410 0x7f746c0cc440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40088/0 (socket says 172.21.15.44:40088) 2024-04-22T06:32:02.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.509+0000 7f746bfff640 1 -- 172.21.15.44:0/2869531619 learned_addr learned my addr 172.21.15.44:0/2869531619 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:32:02.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.509+0000 7f7470fc1640 1 --2- 172.21.15.44:0/2869531619 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f746c151f50 0x7f746c158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:02.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.509+0000 7f74717c2640 1 --2- 172.21.15.44:0/2869531619 >> v2:172.21.15.70:3300/0 conn(0x7f746c1598a0 0x7f746c159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:02.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.509+0000 7f746b7fe640 1 -- 172.21.15.44:0/2869531619 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3839613568 0 0) 0x7f746c15a270 con 0x7f746c0c8410 2024-04-22T06:32:02.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.509+0000 7f746b7fe640 1 -- 172.21.15.44:0/2869531619 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7458003520 con 0x7f746c0c8410 2024-04-22T06:32:02.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.509+0000 7f746b7fe640 1 -- 172.21.15.44:0/2869531619 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1104136091 0 0) 0x7f7458003520 con 0x7f746c0c8410 2024-04-22T06:32:02.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.509+0000 7f746b7fe640 1 -- 172.21.15.44:0/2869531619 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f746c151f50 msgr2=0x7f746c158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:02.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.509+0000 7f746b7fe640 1 --2- 172.21.15.44:0/2869531619 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f746c151f50 0x7f746c158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.509+0000 7f746b7fe640 1 -- 172.21.15.44:0/2869531619 >> v2:172.21.15.70:3300/0 conn(0x7f746c1598a0 msgr2=0x7f746c159d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:02.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.509+0000 7f746b7fe640 1 --2- 172.21.15.44:0/2869531619 >> v2:172.21.15.70:3300/0 conn(0x7f746c1598a0 0x7f746c159d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.509+0000 7f746b7fe640 1 -- 172.21.15.44:0/2869531619 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f746c15ba50 con 0x7f746c0c8410 2024-04-22T06:32:02.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.509+0000 7f746b7fe640 1 -- 172.21.15.44:0/2869531619 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1237339257 0 0) 0x7f7454001670 con 0x7f746c0c8410 2024-04-22T06:32:02.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.509+0000 7f746b7fe640 1 -- 172.21.15.44:0/2869531619 >> v1:172.21.15.44:6789/0 conn(0x7f746c0c8410 legacy=0x7f746c0cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:02.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.509+0000 7f746b7fe640 1 --2- 172.21.15.44:0/2869531619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74580039b0 0x7f7458003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:02.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.509+0000 7f746b7fe640 1 --2- 172.21.15.44:0/2869531619 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f746c151f50 0x7f7458004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:02.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.509+0000 7f746b7fe640 1 --2- 172.21.15.44:0/2869531619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7458004d40 0x7f74580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:02.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.509+0000 7f746b7fe640 1 -- 172.21.15.44:0/2869531619 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f746c158db0 con 0x7f74580039b0 2024-04-22T06:32:02.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.509+0000 7f746b7fe640 1 -- 172.21.15.44:0/2869531619 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f746c15b770 con 0x7f7458004d40 2024-04-22T06:32:02.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.509+0000 7f746b7fe640 1 -- 172.21.15.44:0/2869531619 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f745800dbd0 con 0x7f746c151f50 2024-04-22T06:32:02.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.510+0000 7f7470fc1640 1 --2- 172.21.15.44:0/2869531619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74580039b0 0x7f7458003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:02.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.510+0000 7f746bfff640 1 --2- 172.21.15.44:0/2869531619 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f746c151f50 0x7f7458004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:02.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.510+0000 7f74717c2640 1 --2- 172.21.15.44:0/2869531619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7458004d40 0x7f74580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:02.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.510+0000 7f746bfff640 1 -- 172.21.15.44:0/2869531619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7458004d40 msgr2=0x7f74580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:02.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.510+0000 7f746bfff640 1 --2- 172.21.15.44:0/2869531619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7458004d40 0x7f74580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.510+0000 7f746bfff640 1 -- 172.21.15.44:0/2869531619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74580039b0 msgr2=0x7f7458003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:02.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.510+0000 7f746bfff640 1 --2- 172.21.15.44:0/2869531619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74580039b0 0x7f7458003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.510+0000 7f746bfff640 1 -- 172.21.15.44:0/2869531619 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7458003520 con 0x7f746c151f50 2024-04-22T06:32:02.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.510+0000 7f7470fc1640 1 --2- 172.21.15.44:0/2869531619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74580039b0 0x7f7458003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:32:02.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.511+0000 7f746bfff640 1 --2- 172.21.15.44:0/2869531619 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f746c151f50 0x7f7458004800 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f745400c210 tx=0x7f7458005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e410611c4e7ba5c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:02.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.511+0000 7f746b7fe640 1 -- 172.21.15.44:0/2869531619 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7454007d50 con 0x7f746c151f50 2024-04-22T06:32:02.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.511+0000 7f746b7fe640 1 -- 172.21.15.44:0/2869531619 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7454030070 con 0x7f746c151f50 2024-04-22T06:32:02.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.511+0000 7f746b7fe640 1 -- 172.21.15.44:0/2869531619 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745402cd20 con 0x7f746c151f50 2024-04-22T06:32:02.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.511+0000 7f7473225640 1 -- 172.21.15.44:0/2869531619 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f746c151f50 msgr2=0x7f7458004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:02.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.511+0000 7f7473225640 1 --2- 172.21.15.44:0/2869531619 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f746c151f50 0x7f7458004800 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f745400c210 tx=0x7f7458005830 comp rx=0 tx=0).stop 2024-04-22T06:32:02.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.511+0000 7f74717c2640 1 -- 172.21.15.44:0/2869531619 reap_dead start 2024-04-22T06:32:02.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.511+0000 7f7473225640 1 -- 172.21.15.44:0/2869531619 shutdown_connections 2024-04-22T06:32:02.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.511+0000 7f7473225640 1 -- 172.21.15.44:0/2869531619 >> 172.21.15.44:0/2869531619 conn(0x7f746c06d530 msgr2=0x7f746c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:02.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.511+0000 7f7473225640 1 -- 172.21.15.44:0/2869531619 shutdown_connections 2024-04-22T06:32:02.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.511+0000 7f7473225640 1 -- 172.21.15.44:0/2869531619 wait complete. 2024-04-22T06:32:02.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.513+0000 7f7473225640 1 Processor -- start 2024-04-22T06:32:02.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.513+0000 7f7473225640 1 -- start start 2024-04-22T06:32:02.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.513+0000 7f7473225640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f746c151f50 0x7f746c154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:02.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.513+0000 7f7473225640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f746c1555d0 0x7f746c150580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:02.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.513+0000 7f7473225640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f746c1588b0 0x7f746c155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:02.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.513+0000 7f7473225640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f746c15c2f0 con 0x7f746c151f50 2024-04-22T06:32:02.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.513+0000 7f7473225640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f746c15c010 con 0x7f746c1588b0 2024-04-22T06:32:02.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.513+0000 7f7473225640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f746c159380 con 0x7f746c1555d0 2024-04-22T06:32:02.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.513+0000 7f7470fc1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f746c151f50 0x7f746c154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:02.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.514+0000 7f7470fc1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f746c151f50 0x7f746c154bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52318/0 (socket says 172.21.15.44:52318) 2024-04-22T06:32:02.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.514+0000 7f7470fc1640 1 -- 172.21.15.44:0/863021518 learned_addr learned my addr 172.21.15.44:0/863021518 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:02.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.514+0000 7f746bfff640 1 --2- 172.21.15.44:0/863021518 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f746c1555d0 0x7f746c150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:02.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.514+0000 7f74717c2640 1 --2- 172.21.15.44:0/863021518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f746c1588b0 0x7f746c155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:02.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.514+0000 7f7470fc1640 1 -- 172.21.15.44:0/863021518 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f746c1555d0 msgr2=0x7f746c150580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:02.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.514+0000 7f7470fc1640 1 --2- 172.21.15.44:0/863021518 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f746c1555d0 0x7f746c150580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.514+0000 7f7470fc1640 1 -- 172.21.15.44:0/863021518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f746c1588b0 msgr2=0x7f746c155c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:02.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.514+0000 7f7470fc1640 1 --2- 172.21.15.44:0/863021518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f746c1588b0 0x7f746c155c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.514+0000 7f7470fc1640 1 -- 172.21.15.44:0/863021518 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f746c1f0690 con 0x7f746c151f50 2024-04-22T06:32:02.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.514+0000 7f7470fc1640 1 --2- 172.21.15.44:0/863021518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f746c151f50 0x7f746c154bc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f746400f460 tx=0x7f746400f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:02.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.514+0000 7f74697fa640 1 -- 172.21.15.44:0/863021518 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7464003b90 con 0x7f746c151f50 2024-04-22T06:32:02.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.514+0000 7f74697fa640 1 -- 172.21.15.44:0/863021518 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7464003d30 con 0x7f746c151f50 2024-04-22T06:32:02.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.514+0000 7f7473225640 1 -- 172.21.15.44:0/863021518 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f746c1f0970 con 0x7f746c151f50 2024-04-22T06:32:02.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.514+0000 7f74697fa640 1 -- 172.21.15.44:0/863021518 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7464004260 con 0x7f746c151f50 2024-04-22T06:32:02.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.514+0000 7f7473225640 1 -- 172.21.15.44:0/863021518 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f746c1f1450 con 0x7f746c151f50 2024-04-22T06:32:02.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.515+0000 7f7473225640 1 -- 172.21.15.44:0/863021518 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f746c068cc0 con 0x7f746c151f50 2024-04-22T06:32:02.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.516+0000 7f74697fa640 1 -- 172.21.15.44:0/863021518 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7464004400 con 0x7f746c151f50 2024-04-22T06:32:02.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.516+0000 7f74697fa640 1 --2- 172.21.15.44:0/863021518 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f744c06c8f0 0x7f744c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:02.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.516+0000 7f74697fa640 1 -- 172.21.15.44:0/863021518 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f746408d150 con 0x7f746c151f50 2024-04-22T06:32:02.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.516+0000 7f746bfff640 1 --2- 172.21.15.44:0/863021518 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f744c06c8f0 0x7f744c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:02.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.517+0000 7f746bfff640 1 --2- 172.21.15.44:0/863021518 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f744c06c8f0 0x7f744c06edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7454034000 tx=0x7f7454007000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:02.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.520+0000 7f74697fa640 1 -- 172.21.15.44:0/863021518 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7464019030 con 0x7f746c151f50 2024-04-22T06:32:02.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.744+0000 7f7473225640 1 -- 172.21.15.44:0/863021518 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f746c0c3910 con 0x7f746c151f50 2024-04-22T06:32:02.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.744+0000 7f74697fa640 1 -- 172.21.15.44:0/863021518 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) ==== 53+0+88 (secure 0 0 0) 0x7f7464010040 con 0x7f746c151f50 2024-04-22T06:32:02.743 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-22T06:32:02.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.746+0000 7f7473225640 1 -- 172.21.15.44:0/863021518 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f744c06c8f0 msgr2=0x7f744c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:02.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.746+0000 7f7473225640 1 --2- 172.21.15.44:0/863021518 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f744c06c8f0 0x7f744c06edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7454034000 tx=0x7f7454007000 comp rx=0 tx=0).stop 2024-04-22T06:32:02.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.746+0000 7f7473225640 1 -- 172.21.15.44:0/863021518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f746c151f50 msgr2=0x7f746c154bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:02.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.746+0000 7f7473225640 1 --2- 172.21.15.44:0/863021518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f746c151f50 0x7f746c154bc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f746400f460 tx=0x7f746400f920 comp rx=0 tx=0).stop 2024-04-22T06:32:02.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.747+0000 7f7473225640 1 -- 172.21.15.44:0/863021518 shutdown_connections 2024-04-22T06:32:02.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.747+0000 7f7473225640 1 --2- 172.21.15.44:0/863021518 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f744c06c8f0 0x7f744c06edb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.747+0000 7f7473225640 1 --2- 172.21.15.44:0/863021518 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f746c1555d0 0x7f746c150580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.747+0000 7f7473225640 1 --2- 172.21.15.44:0/863021518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f746c1588b0 0x7f746c155c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.747+0000 7f7473225640 1 --2- 172.21.15.44:0/863021518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f746c151f50 0x7f746c154bc0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:02.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.747+0000 7f7473225640 1 -- 172.21.15.44:0/863021518 >> 172.21.15.44:0/863021518 conn(0x7f746c06d530 msgr2=0x7f746c0c6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:02.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.747+0000 7f7473225640 1 -- 172.21.15.44:0/863021518 shutdown_connections 2024-04-22T06:32:02.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02.747+0000 7f7473225640 1 -- 172.21.15.44:0/863021518 wait complete. 2024-04-22T06:32:02.756 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-22T06:32:02.756 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-22T06:32:02.756 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-04-22T06:32:02.756 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-22T06:32:02.756 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-22T06:32:02.756 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-22T06:32:02.757 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-22T06:32:02.757 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-22T06:32:02.757 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-22T06:32:02.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T06:32:02.757 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-04-22T06:32:02.773 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-22T06:32:02.773 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-22T06:32:02.773 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-04-22T06:32:02.773 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-04-22T06:32:02.773 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-04-22T06:32:02.773 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-04-22T06:32:02.773 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-22T06:32:02.773 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-04-22T06:32:02.773 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-04-22T06:32:02.773 INFO:teuthology.orchestra.run.smithi044.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-22T06:32:02.773 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-04-22T06:32:02.773 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-04-22T06:32:02.773 INFO:teuthology.orchestra.run.smithi044.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-22T06:32:02.774 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-04-22T06:32:02.774 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-04-22T06:32:02.774 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-22T06:32:02.774 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 42668sec preferred_lft 42668sec 2024-04-22T06:32:02.774 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link noprefixroute 2024-04-22T06:32:02.774 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-04-22T06:32:02.774 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-22T06:32:02.774 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-04-22T06:32:02.774 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-04-22T06:32:02.774 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-22T06:32:02.775 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T06:32:02.775 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-04-22T06:32:02.775 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link add name ceph-brx type bridge 2024-04-22T06:32:02.775 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip addr flush dev ceph-brx 2024-04-22T06:32:02.775 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set ceph-brx up 2024-04-22T06:32:02.775 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-22T06:32:02.775 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-04-22T06:32:02.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T06:32:02.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T06:32:02.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T06:32:02.911 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-22T06:32:02.934 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-04-22T06:32:02.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T06:32:02.936 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-04-22T06:32:02.993 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-04-22T06:32:02.993 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-22T06:32:02.993 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-22T06:32:02.993 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-22T06:32:02.993 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-22T06:32:02.993 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T06:32:02.993 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-04-22T06:32:02.993 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-22T06:32:02.993 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-22T06:32:02.994 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-22T06:32:02.994 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-04-22T06:32:03.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T06:32:03.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T06:32:03.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T06:32:03.120 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-04-22T06:32:03.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T06:32:03.136 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list-id 2024-04-22T06:32:03.191 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T06:32:03.191 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-04-22T06:32:03.191 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-22T06:32:03.191 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-22T06:32:03.191 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-04-22T06:32:03.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T06:32:03.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T06:32:03.310 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-22T06:32:03.310 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T06:32:03.310 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-04-22T06:32:03.310 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-22T06:32:03.310 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-22T06:32:03.310 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-22T06:32:03.310 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-22T06:32:03.310 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-22T06:32:03.311 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-04-22T06:32:03.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T06:32:03.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T06:32:03.423 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T06:32:03.423 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-04-22T06:32:03.423 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-04-22T06:32:03.423 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set dev brx.0 master ceph-brx 2024-04-22T06:32:03.423 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-04-22T06:32:03.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T06:32:03.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T06:32:03.500 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-22T06:32:03.500 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-22T06:32:03.500 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-22T06:32:03.515 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-22T06:32:03.515 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-22T06:32:03.515 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-22T06:32:03.570 DEBUG:teuthology.orchestra.run.smithi044:> sudo modprobe fuse 2024-04-22T06:32:03.635 DEBUG:teuthology.orchestra.run.smithi044:> cat /proc/self/mounts | awk '{print $2}' 2024-04-22T06:32:03.692 INFO:teuthology.orchestra.run.smithi044.stdout:/proc 2024-04-22T06:32:03.692 INFO:teuthology.orchestra.run.smithi044.stdout:/sys 2024-04-22T06:32:03.692 INFO:teuthology.orchestra.run.smithi044.stdout:/dev 2024-04-22T06:32:03.692 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/security 2024-04-22T06:32:03.692 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/shm 2024-04-22T06:32:03.692 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/pts 2024-04-22T06:32:03.692 INFO:teuthology.orchestra.run.smithi044.stdout:/run 2024-04-22T06:32:03.692 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/cgroup 2024-04-22T06:32:03.693 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/pstore 2024-04-22T06:32:03.693 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/bpf 2024-04-22T06:32:03.693 INFO:teuthology.orchestra.run.smithi044.stdout:/ 2024-04-22T06:32:03.693 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/selinux 2024-04-22T06:32:03.693 INFO:teuthology.orchestra.run.smithi044.stdout:/proc/sys/fs/binfmt_misc 2024-04-22T06:32:03.693 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/mqueue 2024-04-22T06:32:03.693 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/hugepages 2024-04-22T06:32:03.693 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/debug 2024-04-22T06:32:03.693 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/tracing 2024-04-22T06:32:03.693 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-sysusers.service 2024-04-22T06:32:03.693 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/config 2024-04-22T06:32:03.693 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-22T06:32:03.693 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/fuse/connections 2024-04-22T06:32:03.693 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-sysctl.service 2024-04-22T06:32:03.694 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-22T06:32:03.694 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/nfs/rpc_pipefs 2024-04-22T06:32:03.694 INFO:teuthology.orchestra.run.smithi044.stdout:/run/user/1000 2024-04-22T06:32:03.694 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph 2024-04-22T06:32:03.694 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph/osd/ceph-0 2024-04-22T06:32:03.694 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph/osd/ceph-1 2024-04-22T06:32:03.694 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph/osd/ceph-2 2024-04-22T06:32:03.694 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph/osd/ceph-3 2024-04-22T06:32:03.694 INFO:teuthology.orchestra.run.smithi044.stdout:/run/netns 2024-04-22T06:32:03.694 INFO:teuthology.orchestra.run.smithi044.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-22T06:32:03.694 INFO:teuthology.orchestra.run.smithi044.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-22T06:32:03.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T06:32:03.695 DEBUG:teuthology.orchestra.run.smithi044:> ls /sys/fs/fuse/connections 2024-04-22T06:32:03.747 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-22T06:32:03.748 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-22T06:32:03.790 DEBUG:teuthology.orchestra.run.smithi044:> sudo modprobe fuse 2024-04-22T06:32:03.814 DEBUG:teuthology.orchestra.run.smithi044:> cat /proc/self/mounts | awk '{print $2}' 2024-04-22T06:32:03.871 INFO:teuthology.orchestra.run.smithi044.stdout:/proc 2024-04-22T06:32:03.871 INFO:teuthology.orchestra.run.smithi044.stdout:/sys 2024-04-22T06:32:03.871 INFO:teuthology.orchestra.run.smithi044.stdout:/dev 2024-04-22T06:32:03.871 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/security 2024-04-22T06:32:03.871 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/shm 2024-04-22T06:32:03.871 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/pts 2024-04-22T06:32:03.872 INFO:teuthology.orchestra.run.smithi044.stdout:/run 2024-04-22T06:32:03.872 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/cgroup 2024-04-22T06:32:03.872 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/pstore 2024-04-22T06:32:03.872 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/bpf 2024-04-22T06:32:03.872 INFO:teuthology.orchestra.run.smithi044.stdout:/ 2024-04-22T06:32:03.872 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/selinux 2024-04-22T06:32:03.872 INFO:teuthology.orchestra.run.smithi044.stdout:/proc/sys/fs/binfmt_misc 2024-04-22T06:32:03.872 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/mqueue 2024-04-22T06:32:03.872 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/hugepages 2024-04-22T06:32:03.873 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/debug 2024-04-22T06:32:03.873 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/tracing 2024-04-22T06:32:03.873 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-sysusers.service 2024-04-22T06:32:03.873 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/config 2024-04-22T06:32:03.873 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-22T06:32:03.873 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/fuse/connections 2024-04-22T06:32:03.873 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-sysctl.service 2024-04-22T06:32:03.874 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-22T06:32:03.874 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/nfs/rpc_pipefs 2024-04-22T06:32:03.874 INFO:teuthology.orchestra.run.smithi044.stdout:/run/user/1000 2024-04-22T06:32:03.874 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph 2024-04-22T06:32:03.874 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph/osd/ceph-0 2024-04-22T06:32:03.874 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph/osd/ceph-1 2024-04-22T06:32:03.874 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph/osd/ceph-2 2024-04-22T06:32:03.874 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph/osd/ceph-3 2024-04-22T06:32:03.875 INFO:teuthology.orchestra.run.smithi044.stdout:/run/netns 2024-04-22T06:32:03.875 INFO:teuthology.orchestra.run.smithi044.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-22T06:32:03.875 INFO:teuthology.orchestra.run.smithi044.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-22T06:32:03.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T06:32:03.875 DEBUG:teuthology.orchestra.run.smithi044:> ls /sys/fs/fuse/connections 2024-04-22T06:32:03.877 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi044.stderr:2024-04-22T06:32:03.873+0000 7f2b16435480 -1 init, newargv = 0x55fd91a5a910 newargc=15 2024-04-22T06:32:03.877 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi044.stderr:ceph-fuse[45526]: starting ceph client 2024-04-22T06:32:03.888 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi044.stderr:ceph-fuse[45526]: starting fuse 2024-04-22T06:32:03.892 INFO:teuthology.orchestra.run.smithi044.stdout:43 2024-04-22T06:32:03.893 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-22T06:32:03.893 DEBUG:teuthology.orchestra.run.smithi044:> sudo stdin-killer -- python3 -c ' 2024-04-22T06:32:03.893 DEBUG:teuthology.orchestra.run.smithi044:> import glob 2024-04-22T06:32:03.893 DEBUG:teuthology.orchestra.run.smithi044:> import re 2024-04-22T06:32:03.893 DEBUG:teuthology.orchestra.run.smithi044:> import os 2024-04-22T06:32:03.894 DEBUG:teuthology.orchestra.run.smithi044:> import subprocess 2024-04-22T06:32:03.894 DEBUG:teuthology.orchestra.run.smithi044:> 2024-04-22T06:32:03.894 DEBUG:teuthology.orchestra.run.smithi044:> def _find_admin_socket(client_name): 2024-04-22T06:32:03.894 DEBUG:teuthology.orchestra.run.smithi044:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-22T06:32:03.894 DEBUG:teuthology.orchestra.run.smithi044:> files = glob.glob(asok_path) 2024-04-22T06:32:03.894 DEBUG:teuthology.orchestra.run.smithi044:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-22T06:32:03.895 DEBUG:teuthology.orchestra.run.smithi044:> 2024-04-22T06:32:03.895 DEBUG:teuthology.orchestra.run.smithi044:> # Given a non-glob path, it better be there 2024-04-22T06:32:03.895 DEBUG:teuthology.orchestra.run.smithi044:> if "*" not in asok_path: 2024-04-22T06:32:03.895 DEBUG:teuthology.orchestra.run.smithi044:> assert(len(files) == 1) 2024-04-22T06:32:03.895 DEBUG:teuthology.orchestra.run.smithi044:> return files[0] 2024-04-22T06:32:03.895 DEBUG:teuthology.orchestra.run.smithi044:> 2024-04-22T06:32:03.895 DEBUG:teuthology.orchestra.run.smithi044:> for f in files: 2024-04-22T06:32:03.895 DEBUG:teuthology.orchestra.run.smithi044:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-22T06:32:03.895 DEBUG:teuthology.orchestra.run.smithi044:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-22T06:32:03.895 DEBUG:teuthology.orchestra.run.smithi044:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-22T06:32:03.895 DEBUG:teuthology.orchestra.run.smithi044:> contents = proc_f.read() 2024-04-22T06:32:03.895 DEBUG:teuthology.orchestra.run.smithi044:> if mountpoint in contents: 2024-04-22T06:32:03.895 DEBUG:teuthology.orchestra.run.smithi044:> return f 2024-04-22T06:32:03.895 DEBUG:teuthology.orchestra.run.smithi044:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-22T06:32:03.895 DEBUG:teuthology.orchestra.run.smithi044:> 2024-04-22T06:32:03.896 DEBUG:teuthology.orchestra.run.smithi044:> print(_find_admin_socket("client.0")) 2024-04-22T06:32:03.896 DEBUG:teuthology.orchestra.run.smithi044:> ' 2024-04-22T06:32:04.012 INFO:teuthology.orchestra.run.smithi044.stdout:/var/run/ceph/ceph-client.0.45526.asok 2024-04-22T06:32:04.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T06:32:04.023 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.45526.asok 2024-04-22T06:32:04.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T06:32:04.023 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.45526.asok status 2024-04-22T06:32:04.159 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-22T06:32:04.159 INFO:teuthology.orchestra.run.smithi044.stdout: "metadata": { 2024-04-22T06:32:04.160 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_sha1": "86f7587a5a09af35f0895e1a2d08527638fae697", 2024-04-22T06:32:04.160 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_version": "ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev)", 2024-04-22T06:32:04.160 INFO:teuthology.orchestra.run.smithi044.stdout: "entity_id": "0", 2024-04-22T06:32:04.160 INFO:teuthology.orchestra.run.smithi044.stdout: "hostname": "smithi044", 2024-04-22T06:32:04.160 INFO:teuthology.orchestra.run.smithi044.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-22T06:32:04.160 INFO:teuthology.orchestra.run.smithi044.stdout: "pid": "45526", 2024-04-22T06:32:04.160 INFO:teuthology.orchestra.run.smithi044.stdout: "root": "/" 2024-04-22T06:32:04.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-22T06:32:04.160 INFO:teuthology.orchestra.run.smithi044.stdout: "dentry_count": 0, 2024-04-22T06:32:04.160 INFO:teuthology.orchestra.run.smithi044.stdout: "dentry_pinned_count": 0, 2024-04-22T06:32:04.160 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 5019, 2024-04-22T06:32:04.160 INFO:teuthology.orchestra.run.smithi044.stdout: "inst": { 2024-04-22T06:32:04.160 INFO:teuthology.orchestra.run.smithi044.stdout: "name": { 2024-04-22T06:32:04.161 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "client", 2024-04-22T06:32:04.161 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 5019 2024-04-22T06:32:04.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-22T06:32:04.161 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": { 2024-04-22T06:32:04.161 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-22T06:32:04.161 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "192.168.0.1:0", 2024-04-22T06:32:04.161 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 532601388 2024-04-22T06:32:04.161 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-22T06:32:04.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-22T06:32:04.161 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": { 2024-04-22T06:32:04.161 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-22T06:32:04.161 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "192.168.0.1:0", 2024-04-22T06:32:04.161 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 532601388 2024-04-22T06:32:04.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-22T06:32:04.162 INFO:teuthology.orchestra.run.smithi044.stdout: "inst_str": "client.5019 192.168.0.1:0/532601388", 2024-04-22T06:32:04.162 INFO:teuthology.orchestra.run.smithi044.stdout: "addr_str": "192.168.0.1:0/532601388", 2024-04-22T06:32:04.162 INFO:teuthology.orchestra.run.smithi044.stdout: "inode_count": 1, 2024-04-22T06:32:04.162 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_epoch": 18, 2024-04-22T06:32:04.162 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_epoch": 27, 2024-04-22T06:32:04.162 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_epoch_barrier": 0, 2024-04-22T06:32:04.162 INFO:teuthology.orchestra.run.smithi044.stdout: "blocklisted": false, 2024-04-22T06:32:04.162 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs" 2024-04-22T06:32:04.162 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-22T06:32:04.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T06:32:04.168 DEBUG:teuthology.orchestra.run.smithi044:> stat --file-system '--printf=%T 2024-04-22T06:32:04.168 DEBUG:teuthology.orchestra.run.smithi044:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-22T06:32:04.183 INFO:teuthology.orchestra.run.smithi044.stdout:fuseblk 2024-04-22T06:32:04.183 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-22T06:32:04.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T06:32:04.184 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-22T06:32:04.248 INFO:teuthology.run_tasks:Running task quiescer... 2024-04-22T06:32:04.259 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T06:32:04.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.391+0000 7f45ea37e640 1 Processor -- start 2024-04-22T06:32:04.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.391+0000 7f45ea37e640 1 -- start start 2024-04-22T06:32:04.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.392+0000 7f45ea37e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f45e415ea50 0x7f45e406d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:04.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.392+0000 7f45ea37e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f45e40cd170 0x7f45e40cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:04.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.392+0000 7f45ea37e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45e40cb990 con 0x7f45e415e670 2024-04-22T06:32:04.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.392+0000 7f45ea37e640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f45e415f480 con 0x7f45e40cd170 2024-04-22T06:32:04.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.392+0000 7f45ea37e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f45e41558d0 con 0x7f45e415ea50 2024-04-22T06:32:04.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.392+0000 7f45e891b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f45e40cd170 0x7f45e40cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:04.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.392+0000 7f45e891b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f45e40cd170 0x7f45e40cd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52326/0 (socket says 172.21.15.44:52326) 2024-04-22T06:32:04.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.392+0000 7f45e891b640 1 -- 172.21.15.44:0/989081812 learned_addr learned my addr 172.21.15.44:0/989081812 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:04.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.392+0000 7f45e37fe640 1 --2- 172.21.15.44:0/989081812 >> v2:172.21.15.70:3300/0 conn(0x7f45e415ea50 0x7f45e406d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:04.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.392+0000 7f45e891b640 1 -- 172.21.15.44:0/989081812 >> v2:172.21.15.70:3300/0 conn(0x7f45e415ea50 msgr2=0x7f45e406d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:04.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.392+0000 7f45e891b640 1 --2- 172.21.15.44:0/989081812 >> v2:172.21.15.70:3300/0 conn(0x7f45e415ea50 0x7f45e406d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:04.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.392+0000 7f45e891b640 1 -- 172.21.15.44:0/989081812 >> v1:172.21.15.70:6789/0 conn(0x7f45e415e670 legacy=0x7f45e40c7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:04.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.392+0000 7f45e891b640 1 -- 172.21.15.44:0/989081812 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45e40cc530 con 0x7f45e40cd170 2024-04-22T06:32:04.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.393+0000 7f45e891b640 1 --2- 172.21.15.44:0/989081812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45e40cd170 0x7f45e40cd580 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f45e406d8e0 tx=0x7f45dc02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a610404a530d78 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:04.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.393+0000 7f45e2ffd640 1 -- 172.21.15.44:0/989081812 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45dc02cb20 con 0x7f45e40cd170 2024-04-22T06:32:04.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.393+0000 7f45e2ffd640 1 -- 172.21.15.44:0/989081812 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45dc02ccc0 con 0x7f45e40cd170 2024-04-22T06:32:04.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.393+0000 7f45e2ffd640 1 -- 172.21.15.44:0/989081812 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45dc034610 con 0x7f45e40cd170 2024-04-22T06:32:04.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.394+0000 7f45ea37e640 1 -- 172.21.15.44:0/989081812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45e40cd170 msgr2=0x7f45e40cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:04.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.394+0000 7f45ea37e640 1 --2- 172.21.15.44:0/989081812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45e40cd170 0x7f45e40cd580 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f45e406d8e0 tx=0x7f45dc02bf80 comp rx=0 tx=0).stop 2024-04-22T06:32:04.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.394+0000 7f45ea37e640 1 -- 172.21.15.44:0/989081812 shutdown_connections 2024-04-22T06:32:04.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.394+0000 7f45ea37e640 1 --2- 172.21.15.44:0/989081812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45e40cd170 0x7f45e40cd580 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:04.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.394+0000 7f45ea37e640 1 --2- 172.21.15.44:0/989081812 >> v2:172.21.15.70:3300/0 conn(0x7f45e415ea50 0x7f45e406d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:04.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.394+0000 7f45ea37e640 1 -- 172.21.15.44:0/989081812 >> 172.21.15.44:0/989081812 conn(0x7f45e4072fb0 msgr2=0x7f45e40733c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:04.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.394+0000 7f45ea37e640 1 -- 172.21.15.44:0/989081812 shutdown_connections 2024-04-22T06:32:04.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.394+0000 7f45ea37e640 1 -- 172.21.15.44:0/989081812 wait complete. 2024-04-22T06:32:04.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.395+0000 7f45ea37e640 1 Processor -- start 2024-04-22T06:32:04.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.395+0000 7f45ea37e640 1 -- start start 2024-04-22T06:32:04.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.396+0000 7f45ea37e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45e40cd170 0x7f45e406c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:04.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.396+0000 7f45ea37e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f45e415e670 0x7f45e414fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:04.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.396+0000 7f45ea37e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45e415ea50 0x7f45e4150800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:04.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.396+0000 7f45ea37e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f45e40ccaf0 con 0x7f45e40cd170 2024-04-22T06:32:04.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.396+0000 7f45ea37e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f45e40cbf70 con 0x7f45e415ea50 2024-04-22T06:32:04.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.396+0000 7f45ea37e640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f45e40cbc00 con 0x7f45e415e670 2024-04-22T06:32:04.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.396+0000 7f45e3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45e40cd170 0x7f45e406c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:04.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.396+0000 7f45e3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45e40cd170 0x7f45e406c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52334/0 (socket says 172.21.15.44:52334) 2024-04-22T06:32:04.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.396+0000 7f45e3fff640 1 -- 172.21.15.44:0/1169518051 learned_addr learned my addr 172.21.15.44:0/1169518051 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:04.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.396+0000 7f45e37fe640 1 --2- 172.21.15.44:0/1169518051 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f45e415e670 0x7f45e414fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:04.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.396+0000 7f45e891b640 1 --2- 172.21.15.44:0/1169518051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45e415ea50 0x7f45e4150800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:04.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.396+0000 7f45e3fff640 1 -- 172.21.15.44:0/1169518051 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f45e415e670 msgr2=0x7f45e414fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:04.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.396+0000 7f45e3fff640 1 --2- 172.21.15.44:0/1169518051 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f45e415e670 0x7f45e414fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:04.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.396+0000 7f45e3fff640 1 -- 172.21.15.44:0/1169518051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45e415ea50 msgr2=0x7f45e4150800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:04.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.396+0000 7f45e3fff640 1 --2- 172.21.15.44:0/1169518051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45e415ea50 0x7f45e4150800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:04.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.396+0000 7f45e3fff640 1 -- 172.21.15.44:0/1169518051 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45e41f5110 con 0x7f45e40cd170 2024-04-22T06:32:04.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.396+0000 7f45e891b640 1 --2- 172.21.15.44:0/1169518051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45e415ea50 0x7f45e4150800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:32:04.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.396+0000 7f45e3fff640 1 --2- 172.21.15.44:0/1169518051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45e40cd170 0x7f45e406c8e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f45d800c430 tx=0x7f45d800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:04.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.396+0000 7f45e0ff9640 1 -- 172.21.15.44:0/1169518051 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d80052d0 con 0x7f45e40cd170 2024-04-22T06:32:04.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.396+0000 7f45e0ff9640 1 -- 172.21.15.44:0/1169518051 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45d8005490 con 0x7f45e40cd170 2024-04-22T06:32:04.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.396+0000 7f45ea37e640 1 -- 172.21.15.44:0/1169518051 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45e41f53f0 con 0x7f45e40cd170 2024-04-22T06:32:04.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.396+0000 7f45e0ff9640 1 -- 172.21.15.44:0/1169518051 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d8007260 con 0x7f45e40cd170 2024-04-22T06:32:04.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.397+0000 7f45ea37e640 1 -- 172.21.15.44:0/1169518051 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45e41f15c0 con 0x7f45e40cd170 2024-04-22T06:32:04.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.397+0000 7f45ea37e640 1 -- 172.21.15.44:0/1169518051 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45e4068cc0 con 0x7f45e40cd170 2024-04-22T06:32:04.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.398+0000 7f45e0ff9640 1 -- 172.21.15.44:0/1169518051 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f45d8007400 con 0x7f45e40cd170 2024-04-22T06:32:04.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.398+0000 7f45e0ff9640 1 --2- 172.21.15.44:0/1169518051 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f45d006c8f0 0x7f45d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:04.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.398+0000 7f45e0ff9640 1 -- 172.21.15.44:0/1169518051 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f45d808b4b0 con 0x7f45e40cd170 2024-04-22T06:32:04.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.399+0000 7f45e37fe640 1 --2- 172.21.15.44:0/1169518051 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f45d006c8f0 0x7f45d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:04.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.399+0000 7f45e37fe640 1 --2- 172.21.15.44:0/1169518051 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f45d006c8f0 0x7f45d006edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f45d4006fd0 tx=0x7f45d4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:04.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.403+0000 7f45e0ff9640 1 -- 172.21.15.44:0/1169518051 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f45d80594e0 con 0x7f45e40cd170 2024-04-22T06:32:04.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.590+0000 7f45ea37e640 1 -- 172.21.15.44:0/1169518051 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f45e40cc310 con 0x7f45e40cd170 2024-04-22T06:32:04.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.591+0000 7f45e0ff9640 1 -- 172.21.15.44:0/1169518051 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18794 (secure 0 0 0) 0x7f45d8058de0 con 0x7f45e40cd170 2024-04-22T06:32:04.590 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:32:04.590 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":27,"fsid":"9b0a34ca-403d-4513-8a66-d3749ba559aa","created":"2024-04-22T06:30:42.334581+0000","modified":"2024-04-22T06:31:48.138913+0000","last_up_change":"2024-04-22T06:30:52.942257+0000","last_in_change":"2024-04-22T06:30:44.467727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T06:30:54.560256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T06:31:04.612387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T06:31:05.312124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-22T06:31:09.025550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b4a6692-53c9-472d-b2f3-1dd29a6679ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6817","nonce":2220268493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6819","nonce":2220268493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6823","nonce":2220268493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6821","nonce":2220268493}]},"public_addr":"172.21.15.44:6817/2220268493","cluster_addr":"172.21.15.44:6819/2220268493","heartbeat_back_addr":"172.21.15.44:6823/2220268493","heartbeat_front_addr":"172.21.15.44:6821/2220268493","state":["exists","up"]},{"osd":1,"uuid":"70a6370a-7c0a-4d64-a0e4-b2e726b4f291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6809","nonce":1278678977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6811","nonce":1278678977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6815","nonce":1278678977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6813","nonce":1278678977}]},"public_addr":"172.21.15.44:6809/1278678977","cluster_addr":"172.21.15.44:6811/1278678977","heartbeat_back_addr":"172.21.15.44:6815/1278678977","heartbeat_front_addr":"172.21.15.44:6813/1278678977","state":["exists","up"]},{"osd":2,"uuid":"f2e51a62-ca91-470d-bd5a-5710c61deee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6801","nonce":2544087903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6803","nonce":2544087903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6807","nonce":2544087903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6805","nonce":2544087903}]},"public_addr":"172.21.15.44:6801/2544087903","cluster_addr":"172.21.15.44:6803/2544087903","heartbeat_back_addr":"172.21.15.44:6807/2544087903","heartbeat_front_addr":"172.21.15.44:6805/2544087903","state":["exists","up"]},{"osd":3,"uuid":"bc514203-9af9-4f1f-b18e-765bdd5de6e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6825","nonce":1692928109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6827","nonce":1692928109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6831","nonce":1692928109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6829","nonce":1692928109}]},"public_addr":"172.21.15.44:6825/1692928109","cluster_addr":"172.21.15.44:6827/1692928109","heartbeat_back_addr":"172.21.15.44:6831/1692928109","heartbeat_front_addr":"172.21.15.44:6829/1692928109","state":["exists","up"]},{"osd":4,"uuid":"357c439a-6e08-4c19-ac18-b68e6c890497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6820","nonce":3802577427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6824","nonce":3802577427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6831","nonce":3802577427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6828","nonce":3802577427}]},"public_addr":"172.21.15.70:6820/3802577427","cluster_addr":"172.21.15.70:6824/3802577427","heartbeat_back_addr":"172.21.15.70:6831/3802577427","heartbeat_front_addr":"172.21.15.70:6828/3802577427","state":["exists","up"]},{"osd":5,"uuid":"952fb88f-d56e-4305-bfa8-d32674332dfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6801","nonce":664757345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6803","nonce":664757345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6807","nonce":664757345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6805","nonce":664757345}]},"public_addr":"172.21.15.70:6801/664757345","cluster_addr":"172.21.15.70:6803/664757345","heartbeat_back_addr":"172.21.15.70:6807/664757345","heartbeat_front_addr":"172.21.15.70:6805/664757345","state":["exists","up"]},{"osd":6,"uuid":"74399b3d-fac2-45ab-9172-2cd95b02450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6817","nonce":1623253482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6821","nonce":1623253482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6827","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6829","nonce":1623253482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6825","nonce":1623253482}]},"public_addr":"172.21.15.70:6817/1623253482","cluster_addr":"172.21.15.70:6821/1623253482","heartbeat_back_addr":"172.21.15.70:6829/1623253482","heartbeat_front_addr":"172.21.15.70:6825/1623253482","state":["exists","up"]},{"osd":7,"uuid":"8f35508d-6fcc-43ff-8ee7-0498a7ab3b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6809","nonce":1551180300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6811","nonce":1551180300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6815","nonce":1551180300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6813","nonce":1551180300}]},"public_addr":"172.21.15.70:6809/1551180300","cluster_addr":"172.21.15.70:6811/1551180300","heartbeat_back_addr":"172.21.15.70:6815/1551180300","heartbeat_front_addr":"172.21.15.70:6813/1551180300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.612502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.653756+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.521169+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.649955+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.640231+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.686638+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.642199+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.700360+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T06:32:04.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.593+0000 7f45ea37e640 1 -- 172.21.15.44:0/1169518051 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f45d006c8f0 msgr2=0x7f45d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:04.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.593+0000 7f45ea37e640 1 --2- 172.21.15.44:0/1169518051 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f45d006c8f0 0x7f45d006edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f45d4006fd0 tx=0x7f45d4008040 comp rx=0 tx=0).stop 2024-04-22T06:32:04.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.593+0000 7f45ea37e640 1 -- 172.21.15.44:0/1169518051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45e40cd170 msgr2=0x7f45e406c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:04.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.593+0000 7f45ea37e640 1 --2- 172.21.15.44:0/1169518051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45e40cd170 0x7f45e406c8e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f45d800c430 tx=0x7f45d800c8f0 comp rx=0 tx=0).stop 2024-04-22T06:32:04.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.593+0000 7f45ea37e640 1 -- 172.21.15.44:0/1169518051 shutdown_connections 2024-04-22T06:32:04.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.593+0000 7f45ea37e640 1 --2- 172.21.15.44:0/1169518051 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f45d006c8f0 0x7f45d006edb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:04.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.593+0000 7f45ea37e640 1 --2- 172.21.15.44:0/1169518051 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f45e415e670 0x7f45e414fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:04.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.593+0000 7f45ea37e640 1 --2- 172.21.15.44:0/1169518051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45e415ea50 0x7f45e4150800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:04.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.593+0000 7f45ea37e640 1 --2- 172.21.15.44:0/1169518051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45e40cd170 0x7f45e406c8e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:04.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.593+0000 7f45ea37e640 1 -- 172.21.15.44:0/1169518051 >> 172.21.15.44:0/1169518051 conn(0x7f45e4072fb0 msgr2=0x7f45e40c9190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:04.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.593+0000 7f45ea37e640 1 -- 172.21.15.44:0/1169518051 shutdown_connections 2024-04-22T06:32:04.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.593+0000 7f45ea37e640 1 -- 172.21.15.44:0/1169518051 wait complete. 2024-04-22T06:32:04.603 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T06:30:54.560256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T06:31:04.612387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T06:31:05.312124+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-22T06:31:09.025550+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-22T06:32:04.604 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T06:32:04.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.708+0000 7f1c3f3bd640 1 Processor -- start 2024-04-22T06:32:04.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.709+0000 7f1c3f3bd640 1 -- start start 2024-04-22T06:32:04.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.709+0000 7f1c3f3bd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1c381552e0 0x7f1c380cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:04.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.709+0000 7f1c3f3bd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1c3806d470 0x7f1c380d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:04.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.709+0000 7f1c3f3bd640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c3815ccf0 con 0x7f1c38154f00 2024-04-22T06:32:04.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.709+0000 7f1c3f3bd640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f1c380c6740 con 0x7f1c381552e0 2024-04-22T06:32:04.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.709+0000 7f1c3f3bd640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f1c380c5b30 con 0x7f1c3806d470 2024-04-22T06:32:04.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.710+0000 7f1c3c958640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1c381552e0 0x7f1c380cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:04.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.710+0000 7f1c3d159640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f1c38154f00 0x7f1c3815c5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40096/0 (socket says 172.21.15.44:40096) 2024-04-22T06:32:04.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.710+0000 7f1c3d159640 1 -- 172.21.15.44:0/945162840 learned_addr learned my addr 172.21.15.44:0/945162840 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:32:04.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.710+0000 7f1c3d95a640 1 --2- 172.21.15.44:0/945162840 >> v2:172.21.15.70:3300/0 conn(0x7f1c3806d470 0x7f1c380d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:04.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.710+0000 7f1c3c958640 1 -- 172.21.15.44:0/945162840 >> v2:172.21.15.70:3300/0 conn(0x7f1c3806d470 msgr2=0x7f1c380d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:04.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.710+0000 7f1c3c958640 1 --2- 172.21.15.44:0/945162840 >> v2:172.21.15.70:3300/0 conn(0x7f1c3806d470 0x7f1c380d0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:04.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.710+0000 7f1c3c958640 1 -- 172.21.15.44:0/945162840 >> v1:172.21.15.44:6789/0 conn(0x7f1c38154f00 legacy=0x7f1c3815c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:04.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.710+0000 7f1c2ffff640 1 -- 172.21.15.44:0/945162840 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2830632599 0 0) 0x7f1c3815ccf0 con 0x7f1c38154f00 2024-04-22T06:32:04.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.710+0000 7f1c3c958640 1 -- 172.21.15.44:0/945162840 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c380c5e10 con 0x7f1c381552e0 2024-04-22T06:32:04.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.711+0000 7f1c3c958640 1 --2- 172.21.15.44:0/945162840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c381552e0 0x7f1c380cb430 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1c3806d190 tx=0x7f1c2802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a15feac33ade5816 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:04.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.711+0000 7f1c2ffff640 1 -- 172.21.15.44:0/945162840 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c280315a0 con 0x7f1c381552e0 2024-04-22T06:32:04.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.711+0000 7f1c2ffff640 1 -- 172.21.15.44:0/945162840 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c28037070 con 0x7f1c381552e0 2024-04-22T06:32:04.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.711+0000 7f1c2ffff640 1 -- 172.21.15.44:0/945162840 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c28031960 con 0x7f1c381552e0 2024-04-22T06:32:04.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.711+0000 7f1c3f3bd640 1 -- 172.21.15.44:0/945162840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c381552e0 msgr2=0x7f1c380cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:04.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.711+0000 7f1c3f3bd640 1 --2- 172.21.15.44:0/945162840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c381552e0 0x7f1c380cb430 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1c3806d190 tx=0x7f1c2802b840 comp rx=0 tx=0).stop 2024-04-22T06:32:04.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.711+0000 7f1c3f3bd640 1 -- 172.21.15.44:0/945162840 shutdown_connections 2024-04-22T06:32:04.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.711+0000 7f1c3f3bd640 1 --2- 172.21.15.44:0/945162840 >> v2:172.21.15.70:3300/0 conn(0x7f1c3806d470 0x7f1c380d0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:04.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.711+0000 7f1c3f3bd640 1 --2- 172.21.15.44:0/945162840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c381552e0 0x7f1c380cb430 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:04.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.711+0000 7f1c3f3bd640 1 -- 172.21.15.44:0/945162840 >> 172.21.15.44:0/945162840 conn(0x7f1c380733c0 msgr2=0x7f1c380737d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:04.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.711+0000 7f1c3f3bd640 1 -- 172.21.15.44:0/945162840 shutdown_connections 2024-04-22T06:32:04.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.711+0000 7f1c3f3bd640 1 -- 172.21.15.44:0/945162840 wait complete. 2024-04-22T06:32:04.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.713+0000 7f1c3f3bd640 1 Processor -- start 2024-04-22T06:32:04.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.713+0000 7f1c3f3bd640 1 -- start start 2024-04-22T06:32:04.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.713+0000 7f1c3f3bd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c3806d470 0x7f1c3814c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:04.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.713+0000 7f1c3f3bd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c38154f00 0x7f1c381f48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:04.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.713+0000 7f1c3f3bd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1c381552e0 0x7f1c381498b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:04.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.713+0000 7f1c3f3bd640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1c380c6bd0 con 0x7f1c38154f00 2024-04-22T06:32:04.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.713+0000 7f1c3f3bd640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f1c380c63d0 con 0x7f1c3806d470 2024-04-22T06:32:04.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.713+0000 7f1c3f3bd640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f1c380c6d50 con 0x7f1c381552e0 2024-04-22T06:32:04.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.713+0000 7f1c3c958640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c38154f00 0x7f1c381f48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:04.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.713+0000 7f1c3c958640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c38154f00 0x7f1c381f48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52342/0 (socket says 172.21.15.44:52342) 2024-04-22T06:32:04.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.713+0000 7f1c3c958640 1 -- 172.21.15.44:0/3578777494 learned_addr learned my addr 172.21.15.44:0/3578777494 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:04.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.714+0000 7f1c3d159640 1 --2- 172.21.15.44:0/3578777494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c3806d470 0x7f1c3814c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:04.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.714+0000 7f1c3d95a640 1 --2- 172.21.15.44:0/3578777494 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1c381552e0 0x7f1c381498b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:04.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.714+0000 7f1c3c958640 1 -- 172.21.15.44:0/3578777494 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1c381552e0 msgr2=0x7f1c381498b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:04.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.714+0000 7f1c3c958640 1 --2- 172.21.15.44:0/3578777494 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1c381552e0 0x7f1c381498b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:04.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.714+0000 7f1c3c958640 1 -- 172.21.15.44:0/3578777494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c3806d470 msgr2=0x7f1c3814c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:04.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.714+0000 7f1c3c958640 1 --2- 172.21.15.44:0/3578777494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c3806d470 0x7f1c3814c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:04.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.714+0000 7f1c3c958640 1 -- 172.21.15.44:0/3578777494 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c38152180 con 0x7f1c38154f00 2024-04-22T06:32:04.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.714+0000 7f1c3c958640 1 --2- 172.21.15.44:0/3578777494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c38154f00 0x7f1c381f48c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1c28000c00 tx=0x7f1c280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:04.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.714+0000 7f1c2dffb640 1 -- 172.21.15.44:0/3578777494 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c28032840 con 0x7f1c38154f00 2024-04-22T06:32:04.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.714+0000 7f1c2dffb640 1 -- 172.21.15.44:0/3578777494 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c28032a00 con 0x7f1c38154f00 2024-04-22T06:32:04.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.714+0000 7f1c3f3bd640 1 -- 172.21.15.44:0/3578777494 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c381f40c0 con 0x7f1c38154f00 2024-04-22T06:32:04.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.714+0000 7f1c2dffb640 1 -- 172.21.15.44:0/3578777494 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c28032df0 con 0x7f1c38154f00 2024-04-22T06:32:04.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.714+0000 7f1c3f3bd640 1 -- 172.21.15.44:0/3578777494 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c380cbdc0 con 0x7f1c38154f00 2024-04-22T06:32:04.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.715+0000 7f1c3f3bd640 1 -- 172.21.15.44:0/3578777494 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c38068cc0 con 0x7f1c38154f00 2024-04-22T06:32:04.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.716+0000 7f1c2dffb640 1 -- 172.21.15.44:0/3578777494 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1c2804d3f0 con 0x7f1c38154f00 2024-04-22T06:32:04.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.716+0000 7f1c2dffb640 1 --2- 172.21.15.44:0/3578777494 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f1c1c06c920 0x7f1c1c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:04.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.716+0000 7f1c2dffb640 1 -- 172.21.15.44:0/3578777494 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f1c2802f030 con 0x7f1c38154f00 2024-04-22T06:32:04.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.717+0000 7f1c3d159640 1 --2- 172.21.15.44:0/3578777494 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f1c1c06c920 0x7f1c1c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:04.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.717+0000 7f1c3d159640 1 --2- 172.21.15.44:0/3578777494 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f1c1c06c920 0x7f1c1c06ede0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1c34000f60 tx=0x7f1c34007b50 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:04.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.721+0000 7f1c2dffb640 1 -- 172.21.15.44:0/3578777494 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1c28040070 con 0x7f1c38154f00 2024-04-22T06:32:04.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.906+0000 7f1c3f3bd640 1 -- 172.21.15.44:0/3578777494 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1c380caee0 con 0x7f1c38154f00 2024-04-22T06:32:04.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.906+0000 7f1c2dffb640 1 -- 172.21.15.44:0/3578777494 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f1c2807bb90 con 0x7f1c38154f00 2024-04-22T06:32:04.905 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-04-22T06:32:04.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.908+0000 7f1c3f3bd640 1 -- 172.21.15.44:0/3578777494 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f1c1c06c920 msgr2=0x7f1c1c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:04.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.908+0000 7f1c3f3bd640 1 --2- 172.21.15.44:0/3578777494 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f1c1c06c920 0x7f1c1c06ede0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1c34000f60 tx=0x7f1c34007b50 comp rx=0 tx=0).stop 2024-04-22T06:32:04.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.908+0000 7f1c3f3bd640 1 -- 172.21.15.44:0/3578777494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c38154f00 msgr2=0x7f1c381f48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:04.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.908+0000 7f1c3f3bd640 1 --2- 172.21.15.44:0/3578777494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c38154f00 0x7f1c381f48c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1c28000c00 tx=0x7f1c280046c0 comp rx=0 tx=0).stop 2024-04-22T06:32:04.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.908+0000 7f1c3f3bd640 1 -- 172.21.15.44:0/3578777494 shutdown_connections 2024-04-22T06:32:04.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.908+0000 7f1c3f3bd640 1 --2- 172.21.15.44:0/3578777494 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f1c1c06c920 0x7f1c1c06ede0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:04.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.908+0000 7f1c3f3bd640 1 --2- 172.21.15.44:0/3578777494 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1c381552e0 0x7f1c381498b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:04.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.908+0000 7f1c3f3bd640 1 --2- 172.21.15.44:0/3578777494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c38154f00 0x7f1c381f48c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:04.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.908+0000 7f1c3f3bd640 1 --2- 172.21.15.44:0/3578777494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c3806d470 0x7f1c3814c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:04.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.908+0000 7f1c3f3bd640 1 -- 172.21.15.44:0/3578777494 >> 172.21.15.44:0/3578777494 conn(0x7f1c380733c0 msgr2=0x7f1c381525f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:04.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.908+0000 7f1c3f3bd640 1 -- 172.21.15.44:0/3578777494 shutdown_connections 2024-04-22T06:32:04.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:04.908+0000 7f1c3f3bd640 1 -- 172.21.15.44:0/3578777494 wait complete. 2024-04-22T06:32:04.918 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T06:32:05.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.023+0000 7fb4f6448640 1 Processor -- start 2024-04-22T06:32:05.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.023+0000 7fb4f6448640 1 -- start start 2024-04-22T06:32:05.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.024+0000 7fb4f6448640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb4f0155460 0x7fb4f015f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:05.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.024+0000 7fb4f6448640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb4f00cc6d0 0x7fb4f0073230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:05.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.024+0000 7fb4f6448640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4f00cd0a0 con 0x7fb4f00ccb40 2024-04-22T06:32:05.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.024+0000 7fb4f6448640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fb4f0155840 con 0x7fb4f0155460 2024-04-22T06:32:05.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.024+0000 7fb4f6448640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb4f00d1520 con 0x7fb4f00cc6d0 2024-04-22T06:32:05.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.024+0000 7fb4effff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb4f0155460 0x7fb4f015f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:05.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.024+0000 7fb4f49e5640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fb4f00ccb40 0x7fb4f00c5c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40108/0 (socket says 172.21.15.44:40108) 2024-04-22T06:32:05.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.024+0000 7fb4f49e5640 1 -- 172.21.15.44:0/919905935 learned_addr learned my addr 172.21.15.44:0/919905935 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:32:05.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.024+0000 7fb4ef7fe640 1 --2- 172.21.15.44:0/919905935 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb4f00cc6d0 0x7fb4f0073230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:05.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.024+0000 7fb4effff640 1 -- 172.21.15.44:0/919905935 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb4f00cc6d0 msgr2=0x7fb4f0073230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:05.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.024+0000 7fb4effff640 1 --2- 172.21.15.44:0/919905935 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb4f00cc6d0 0x7fb4f0073230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:05.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.024+0000 7fb4effff640 1 -- 172.21.15.44:0/919905935 >> v1:172.21.15.44:6789/0 conn(0x7fb4f00ccb40 legacy=0x7fb4f00c5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:05.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.024+0000 7fb4eeffd640 1 -- 172.21.15.44:0/919905935 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1620163162 0 0) 0x7fb4f00cd0a0 con 0x7fb4f00ccb40 2024-04-22T06:32:05.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.024+0000 7fb4effff640 1 -- 172.21.15.44:0/919905935 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4f00d0910 con 0x7fb4f0155460 2024-04-22T06:32:05.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.025+0000 7fb4effff640 1 --2- 172.21.15.44:0/919905935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4f0155460 0x7fb4f015f050 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb4d80097a0 tx=0x7fb4d802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37205aae7dc23dc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:05.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.025+0000 7fb4eeffd640 1 -- 172.21.15.44:0/919905935 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d80315a0 con 0x7fb4f0155460 2024-04-22T06:32:05.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.025+0000 7fb4eeffd640 1 -- 172.21.15.44:0/919905935 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4d8037070 con 0x7fb4f0155460 2024-04-22T06:32:05.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.025+0000 7fb4eeffd640 1 -- 172.21.15.44:0/919905935 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8031960 con 0x7fb4f0155460 2024-04-22T06:32:05.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.026+0000 7fb4f6448640 1 -- 172.21.15.44:0/919905935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4f0155460 msgr2=0x7fb4f015f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:05.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.026+0000 7fb4f6448640 1 --2- 172.21.15.44:0/919905935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4f0155460 0x7fb4f015f050 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb4d80097a0 tx=0x7fb4d802b840 comp rx=0 tx=0).stop 2024-04-22T06:32:05.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.026+0000 7fb4f6448640 1 -- 172.21.15.44:0/919905935 shutdown_connections 2024-04-22T06:32:05.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.026+0000 7fb4f6448640 1 --2- 172.21.15.44:0/919905935 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb4f00cc6d0 0x7fb4f0073230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:05.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.026+0000 7fb4f6448640 1 --2- 172.21.15.44:0/919905935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4f0155460 0x7fb4f015f050 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:05.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.026+0000 7fb4f6448640 1 -- 172.21.15.44:0/919905935 >> 172.21.15.44:0/919905935 conn(0x7fb4f006cca0 msgr2=0x7fb4f006d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:05.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.026+0000 7fb4f6448640 1 -- 172.21.15.44:0/919905935 shutdown_connections 2024-04-22T06:32:05.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.026+0000 7fb4f6448640 1 -- 172.21.15.44:0/919905935 wait complete. 2024-04-22T06:32:05.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.027+0000 7fb4f6448640 1 Processor -- start 2024-04-22T06:32:05.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.027+0000 7fb4f6448640 1 -- start start 2024-04-22T06:32:05.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.028+0000 7fb4f6448640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4f00cc6d0 0x7fb4f0148a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:05.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.028+0000 7fb4f6448640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4f00ccb40 0x7fb4f0157c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:05.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.028+0000 7fb4f6448640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb4f0155460 0x7fb4f014a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:05.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.028+0000 7fb4f6448640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f00d11b0 con 0x7fb4f00cc6d0 2024-04-22T06:32:05.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.028+0000 7fb4f6448640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f00d0ed0 con 0x7fb4f00ccb40 2024-04-22T06:32:05.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.028+0000 7fb4f6448640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb4f00c6310 con 0x7fb4f0155460 2024-04-22T06:32:05.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.028+0000 7fb4effff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4f00cc6d0 0x7fb4f0148a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:05.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.028+0000 7fb4effff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4f00cc6d0 0x7fb4f0148a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52348/0 (socket says 172.21.15.44:52348) 2024-04-22T06:32:05.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.028+0000 7fb4effff640 1 -- 172.21.15.44:0/2017856910 learned_addr learned my addr 172.21.15.44:0/2017856910 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:05.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.028+0000 7fb4ef7fe640 1 --2- 172.21.15.44:0/2017856910 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4f00ccb40 0x7fb4f0157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:05.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.028+0000 7fb4f49e5640 1 --2- 172.21.15.44:0/2017856910 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb4f0155460 0x7fb4f014a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:05.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.028+0000 7fb4effff640 1 -- 172.21.15.44:0/2017856910 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb4f0155460 msgr2=0x7fb4f014a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:05.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.028+0000 7fb4effff640 1 --2- 172.21.15.44:0/2017856910 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb4f0155460 0x7fb4f014a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:05.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.028+0000 7fb4effff640 1 -- 172.21.15.44:0/2017856910 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4f00ccb40 msgr2=0x7fb4f0157c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:05.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.028+0000 7fb4effff640 1 --2- 172.21.15.44:0/2017856910 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4f00ccb40 0x7fb4f0157c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:05.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.028+0000 7fb4effff640 1 -- 172.21.15.44:0/2017856910 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4f0148fb0 con 0x7fb4f00cc6d0 2024-04-22T06:32:05.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.028+0000 7fb4f49e5640 1 --2- 172.21.15.44:0/2017856910 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb4f0155460 0x7fb4f014a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:32:05.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.029+0000 7fb4effff640 1 --2- 172.21.15.44:0/2017856910 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4f00cc6d0 0x7fb4f0148a70 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb4d80097a0 tx=0x7fb4d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:05.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.029+0000 7fb4ecff9640 1 -- 172.21.15.44:0/2017856910 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8004230 con 0x7fb4f00cc6d0 2024-04-22T06:32:05.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.029+0000 7fb4ecff9640 1 -- 172.21.15.44:0/2017856910 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4d80043d0 con 0x7fb4f00cc6d0 2024-04-22T06:32:05.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.029+0000 7fb4f6448640 1 -- 172.21.15.44:0/2017856910 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4f01565b0 con 0x7fb4f00cc6d0 2024-04-22T06:32:05.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.029+0000 7fb4ecff9640 1 -- 172.21.15.44:0/2017856910 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8003850 con 0x7fb4f00cc6d0 2024-04-22T06:32:05.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.029+0000 7fb4f6448640 1 -- 172.21.15.44:0/2017856910 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4f00c8440 con 0x7fb4f00cc6d0 2024-04-22T06:32:05.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.030+0000 7fb4f6448640 1 -- 172.21.15.44:0/2017856910 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4f0068cc0 con 0x7fb4f00cc6d0 2024-04-22T06:32:05.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.030+0000 7fb4ecff9640 1 -- 172.21.15.44:0/2017856910 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb4d80039f0 con 0x7fb4f00cc6d0 2024-04-22T06:32:05.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.031+0000 7fb4ecff9640 1 --2- 172.21.15.44:0/2017856910 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb4d006c920 0x7fb4d006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:05.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.031+0000 7fb4ecff9640 1 -- 172.21.15.44:0/2017856910 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fb4d802f030 con 0x7fb4f00cc6d0 2024-04-22T06:32:05.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.031+0000 7fb4ef7fe640 1 --2- 172.21.15.44:0/2017856910 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb4d006c920 0x7fb4d006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:05.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.031+0000 7fb4ef7fe640 1 --2- 172.21.15.44:0/2017856910 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb4d006c920 0x7fb4d006ede0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb4f01f21d0 tx=0x7fb4e0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:05.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.035+0000 7fb4ecff9640 1 -- 172.21.15.44:0/2017856910 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb4d8041070 con 0x7fb4f00cc6d0 2024-04-22T06:32:05.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.219+0000 7fb4f6448640 1 -- 172.21.15.44:0/2017856910 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb4f014f4e0 con 0x7fb4f00cc6d0 2024-04-22T06:32:05.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.220+0000 7fb4ecff9640 1 -- 172.21.15.44:0/2017856910 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fb4d8044340 con 0x7fb4f00cc6d0 2024-04-22T06:32:05.219 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-04-22T06:32:05.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.222+0000 7fb4f6448640 1 -- 172.21.15.44:0/2017856910 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb4d006c920 msgr2=0x7fb4d006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:05.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.222+0000 7fb4f6448640 1 --2- 172.21.15.44:0/2017856910 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb4d006c920 0x7fb4d006ede0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb4f01f21d0 tx=0x7fb4e0008040 comp rx=0 tx=0).stop 2024-04-22T06:32:05.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.222+0000 7fb4f6448640 1 -- 172.21.15.44:0/2017856910 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4f00cc6d0 msgr2=0x7fb4f0148a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:05.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.222+0000 7fb4f6448640 1 --2- 172.21.15.44:0/2017856910 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4f00cc6d0 0x7fb4f0148a70 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb4d80097a0 tx=0x7fb4d80046c0 comp rx=0 tx=0).stop 2024-04-22T06:32:05.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.222+0000 7fb4f6448640 1 -- 172.21.15.44:0/2017856910 shutdown_connections 2024-04-22T06:32:05.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.222+0000 7fb4f6448640 1 --2- 172.21.15.44:0/2017856910 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb4d006c920 0x7fb4d006ede0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:05.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.222+0000 7fb4f6448640 1 --2- 172.21.15.44:0/2017856910 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb4f0155460 0x7fb4f014a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:05.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.222+0000 7fb4f6448640 1 --2- 172.21.15.44:0/2017856910 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4f00ccb40 0x7fb4f0157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:05.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.222+0000 7fb4f6448640 1 --2- 172.21.15.44:0/2017856910 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4f00cc6d0 0x7fb4f0148a70 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:05.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.222+0000 7fb4f6448640 1 -- 172.21.15.44:0/2017856910 >> 172.21.15.44:0/2017856910 conn(0x7fb4f006cca0 msgr2=0x7fb4f0073230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:05.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.222+0000 7fb4f6448640 1 -- 172.21.15.44:0/2017856910 shutdown_connections 2024-04-22T06:32:05.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.222+0000 7fb4f6448640 1 -- 172.21.15.44:0/2017856910 wait complete. 2024-04-22T06:32:05.231 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T06:32:05.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.336+0000 7f0b9a0b3640 1 Processor -- start 2024-04-22T06:32:05.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.336+0000 7f0b9a0b3640 1 -- start start 2024-04-22T06:32:05.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.337+0000 7f0b9a0b3640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f0b940ce150 0x7f0b941599a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:05.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.337+0000 7f0b9a0b3640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b9415c690 con 0x7f0b9406fbf0 2024-04-22T06:32:05.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.337+0000 7f0b9a0b3640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b9415c930 con 0x7f0b940c4ce0 2024-04-22T06:32:05.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.337+0000 7f0b9a0b3640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f0b9415cbf0 con 0x7f0b940ce150 2024-04-22T06:32:05.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.337+0000 7f0b92ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f0b940ce150 0x7f0b941599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:05.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.337+0000 7f0b93fff640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f0b9406fbf0 0x7f0b94070060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40114/0 (socket says 172.21.15.44:40114) 2024-04-22T06:32:05.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.337+0000 7f0b93fff640 1 -- 172.21.15.44:0/1368510748 learned_addr learned my addr 172.21.15.44:0/1368510748 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:32:05.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.338+0000 7f0b92ffd640 1 -- 172.21.15.44:0/1368510748 >> v1:172.21.15.70:6789/0 conn(0x7f0b940c4ce0 legacy=0x7f0b940c50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:05.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.338+0000 7f0b92ffd640 1 -- 172.21.15.44:0/1368510748 >> v1:172.21.15.44:6789/0 conn(0x7f0b9406fbf0 legacy=0x7f0b94070060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:05.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.338+0000 7f0b92ffd640 1 -- 172.21.15.44:0/1368510748 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b9415ced0 con 0x7f0b940ce150 2024-04-22T06:32:05.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.338+0000 7f0b927fc640 1 -- 172.21.15.44:0/1368510748 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 79125449 0 0) 0x7f0b9415c690 con 0x7f0b9406fbf0 2024-04-22T06:32:05.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.338+0000 7f0b92ffd640 1 --2- 172.21.15.44:0/1368510748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b940ce150 0x7f0b941599a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0b880097a0 tx=0x7f0b8802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdef53367ec654fb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:05.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.339+0000 7f0b927fc640 1 -- 172.21.15.44:0/1368510748 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b88035020 con 0x7f0b940ce150 2024-04-22T06:32:05.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.339+0000 7f0b927fc640 1 -- 172.21.15.44:0/1368510748 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b88031830 con 0x7f0b940ce150 2024-04-22T06:32:05.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.339+0000 7f0b927fc640 1 -- 172.21.15.44:0/1368510748 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b88031c00 con 0x7f0b940ce150 2024-04-22T06:32:05.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.339+0000 7f0b9a0b3640 1 -- 172.21.15.44:0/1368510748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b940ce150 msgr2=0x7f0b941599a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:05.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.339+0000 7f0b9a0b3640 1 --2- 172.21.15.44:0/1368510748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b940ce150 0x7f0b941599a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0b880097a0 tx=0x7f0b8802b8d0 comp rx=0 tx=0).stop 2024-04-22T06:32:05.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.339+0000 7f0b9a0b3640 1 -- 172.21.15.44:0/1368510748 shutdown_connections 2024-04-22T06:32:05.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.339+0000 7f0b9a0b3640 1 --2- 172.21.15.44:0/1368510748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b940ce150 0x7f0b941599a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:05.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.339+0000 7f0b9a0b3640 1 -- 172.21.15.44:0/1368510748 >> 172.21.15.44:0/1368510748 conn(0x7f0b940cd630 msgr2=0x7f0b940cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:05.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.339+0000 7f0b9a0b3640 1 -- 172.21.15.44:0/1368510748 shutdown_connections 2024-04-22T06:32:05.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.339+0000 7f0b9a0b3640 1 -- 172.21.15.44:0/1368510748 wait complete. 2024-04-22T06:32:05.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.340+0000 7f0b9a0b3640 1 Processor -- start 2024-04-22T06:32:05.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.340+0000 7f0b9a0b3640 1 -- start start 2024-04-22T06:32:05.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.341+0000 7f0b9a0b3640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0b9406fbf0 0x7f0b941540e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:05.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.341+0000 7f0b9a0b3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b940c4ce0 0x7f0b9415b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:05.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.341+0000 7f0b9a0b3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b940ce150 0x7f0b94149280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:05.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.341+0000 7f0b9a0b3640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0b94163c30 con 0x7f0b940ce150 2024-04-22T06:32:05.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.341+0000 7f0b9a0b3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f0b94163ab0 con 0x7f0b940c4ce0 2024-04-22T06:32:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.341+0000 7f0b9a0b3640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f0b94163db0 con 0x7f0b9406fbf0 2024-04-22T06:32:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.341+0000 7f0b93fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b940ce150 0x7f0b94149280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.341+0000 7f0b93fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b940ce150 0x7f0b94149280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52362/0 (socket says 172.21.15.44:52362) 2024-04-22T06:32:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.341+0000 7f0b93fff640 1 -- 172.21.15.44:0/817479751 learned_addr learned my addr 172.21.15.44:0/817479751 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.341+0000 7f0b92ffd640 1 --2- 172.21.15.44:0/817479751 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b940c4ce0 0x7f0b9415b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.341+0000 7f0b937fe640 1 --2- 172.21.15.44:0/817479751 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0b9406fbf0 0x7f0b941540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.342+0000 7f0b93fff640 1 -- 172.21.15.44:0/817479751 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0b9406fbf0 msgr2=0x7f0b941540e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.342+0000 7f0b93fff640 1 --2- 172.21.15.44:0/817479751 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0b9406fbf0 0x7f0b941540e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.342+0000 7f0b93fff640 1 -- 172.21.15.44:0/817479751 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b940c4ce0 msgr2=0x7f0b9415b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.342+0000 7f0b93fff640 1 --2- 172.21.15.44:0/817479751 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b940c4ce0 0x7f0b9415b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.342+0000 7f0b93fff640 1 -- 172.21.15.44:0/817479751 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b9414b500 con 0x7f0b940ce150 2024-04-22T06:32:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.342+0000 7f0b92ffd640 1 --2- 172.21.15.44:0/817479751 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b940c4ce0 0x7f0b9415b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:32:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.342+0000 7f0b937fe640 1 --2- 172.21.15.44:0/817479751 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0b9406fbf0 0x7f0b941540e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:32:05.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.342+0000 7f0b93fff640 1 --2- 172.21.15.44:0/817479751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b940ce150 0x7f0b94149280 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c00c810 tx=0x7f0b8c00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:05.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.342+0000 7f0b7bfff640 1 -- 172.21.15.44:0/817479751 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c018070 con 0x7f0b940ce150 2024-04-22T06:32:05.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.342+0000 7f0b9a0b3640 1 -- 172.21.15.44:0/817479751 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b940cd160 con 0x7f0b940ce150 2024-04-22T06:32:05.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.342+0000 7f0b7bfff640 1 -- 172.21.15.44:0/817479751 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b8c00a640 con 0x7f0b940ce150 2024-04-22T06:32:05.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.342+0000 7f0b7bfff640 1 -- 172.21.15.44:0/817479751 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c00aa30 con 0x7f0b940ce150 2024-04-22T06:32:05.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.342+0000 7f0b9a0b3640 1 -- 172.21.15.44:0/817479751 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b9414ba80 con 0x7f0b940ce150 2024-04-22T06:32:05.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.343+0000 7f0b9a0b3640 1 -- 172.21.15.44:0/817479751 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b94069310 con 0x7f0b940ce150 2024-04-22T06:32:05.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.343+0000 7f0b7bfff640 1 -- 172.21.15.44:0/817479751 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b8c00abd0 con 0x7f0b940ce150 2024-04-22T06:32:05.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.344+0000 7f0b7bfff640 1 --2- 172.21.15.44:0/817479751 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f0b8006c920 0x7f0b8006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:05.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.344+0000 7f0b7bfff640 1 -- 172.21.15.44:0/817479751 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f0b8c08b4f0 con 0x7f0b940ce150 2024-04-22T06:32:05.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.344+0000 7f0b937fe640 1 --2- 172.21.15.44:0/817479751 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f0b8006c920 0x7f0b8006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:05.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.345+0000 7f0b937fe640 1 --2- 172.21.15.44:0/817479751 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f0b8006c920 0x7f0b8006ede0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0b84003eb0 tx=0x7f0b84009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:05.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.348+0000 7f0b7bfff640 1 -- 172.21.15.44:0/817479751 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0b8c0594a0 con 0x7f0b940ce150 2024-04-22T06:32:05.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.537+0000 7f0b9a0b3640 1 -- 172.21.15.44:0/817479751 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0b94148770 con 0x7f0b940ce150 2024-04-22T06:32:05.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.538+0000 7f0b7bfff640 1 -- 172.21.15.44:0/817479751 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f0b8c058da0 con 0x7f0b940ce150 2024-04-22T06:32:05.537 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-04-22T06:32:05.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.539+0000 7f0b9a0b3640 1 -- 172.21.15.44:0/817479751 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f0b8006c920 msgr2=0x7f0b8006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:05.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.539+0000 7f0b9a0b3640 1 --2- 172.21.15.44:0/817479751 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f0b8006c920 0x7f0b8006ede0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0b84003eb0 tx=0x7f0b84009040 comp rx=0 tx=0).stop 2024-04-22T06:32:05.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.539+0000 7f0b9a0b3640 1 -- 172.21.15.44:0/817479751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b940ce150 msgr2=0x7f0b94149280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:05.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.539+0000 7f0b9a0b3640 1 --2- 172.21.15.44:0/817479751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b940ce150 0x7f0b94149280 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c00c810 tx=0x7f0b8c00ccd0 comp rx=0 tx=0).stop 2024-04-22T06:32:05.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.540+0000 7f0b9a0b3640 1 -- 172.21.15.44:0/817479751 shutdown_connections 2024-04-22T06:32:05.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.540+0000 7f0b9a0b3640 1 --2- 172.21.15.44:0/817479751 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f0b8006c920 0x7f0b8006ede0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:05.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.540+0000 7f0b9a0b3640 1 --2- 172.21.15.44:0/817479751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b940ce150 0x7f0b94149280 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:05.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.540+0000 7f0b9a0b3640 1 --2- 172.21.15.44:0/817479751 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b940c4ce0 0x7f0b9415b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:05.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.540+0000 7f0b9a0b3640 1 --2- 172.21.15.44:0/817479751 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0b9406fbf0 0x7f0b941540e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:05.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.540+0000 7f0b9a0b3640 1 -- 172.21.15.44:0/817479751 >> 172.21.15.44:0/817479751 conn(0x7f0b940cd630 msgr2=0x7f0b940c8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:05.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.540+0000 7f0b9a0b3640 1 -- 172.21.15.44:0/817479751 shutdown_connections 2024-04-22T06:32:05.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.540+0000 7f0b9a0b3640 1 -- 172.21.15.44:0/817479751 wait complete. 2024-04-22T06:32:05.549 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-22T06:32:05.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.654+0000 7f118a57e640 1 Processor -- start 2024-04-22T06:32:05.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.654+0000 7f118a57e640 1 -- start start 2024-04-22T06:32:05.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.655+0000 7f118a57e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11841551e0 0x7f11840ce790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:05.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.655+0000 7f118a57e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f118415d180 0x7f11840cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:05.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.655+0000 7f118a57e640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f118415b060 con 0x7f11841555c0 2024-04-22T06:32:05.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.655+0000 7f118a57e640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f118415b300 con 0x7f118415d180 2024-04-22T06:32:05.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.655+0000 7f118a57e640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f118406c460 con 0x7f11841551e0 2024-04-22T06:32:05.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.655+0000 7f1188b1b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f118415d180 0x7f11840cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:05.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.655+0000 7f1188b1b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f118415d180 0x7f11840cecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52374/0 (socket says 172.21.15.44:52374) 2024-04-22T06:32:05.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.655+0000 7f11837fe640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f11841555c0 0x7f118415ca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40124/0 (socket says 172.21.15.44:40124) 2024-04-22T06:32:05.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.655+0000 7f1188b1b640 1 -- 172.21.15.44:0/3696302619 learned_addr learned my addr 172.21.15.44:0/3696302619 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:05.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.655+0000 7f1183fff640 1 --2- 172.21.15.44:0/3696302619 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11841551e0 0x7f11840ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:05.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.655+0000 7f1188b1b640 1 -- 172.21.15.44:0/3696302619 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11841551e0 msgr2=0x7f11840ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:05.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.655+0000 7f1188b1b640 1 --2- 172.21.15.44:0/3696302619 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11841551e0 0x7f11840ce790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:05.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.655+0000 7f1188b1b640 1 -- 172.21.15.44:0/3696302619 >> v1:172.21.15.44:6789/0 conn(0x7f11841555c0 legacy=0x7f118415ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:05.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.655+0000 7f1188b1b640 1 -- 172.21.15.44:0/3696302619 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f118415b5c0 con 0x7f118415d180 2024-04-22T06:32:05.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.655+0000 7f1182ffd640 1 -- 172.21.15.44:0/3696302619 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 535863142 0 0) 0x7f118415b060 con 0x7f11841555c0 2024-04-22T06:32:05.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.656+0000 7f1188b1b640 1 --2- 172.21.15.44:0/3696302619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f118415d180 0x7f11840cecd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f117c009e50 tx=0x7f117c02bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18ca110a112165f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:05.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.656+0000 7f1182ffd640 1 -- 172.21.15.44:0/3696302619 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117c02cb90 con 0x7f118415d180 2024-04-22T06:32:05.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.656+0000 7f1182ffd640 1 -- 172.21.15.44:0/3696302619 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f117c02cd30 con 0x7f118415d180 2024-04-22T06:32:05.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.656+0000 7f1182ffd640 1 -- 172.21.15.44:0/3696302619 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117c034610 con 0x7f118415d180 2024-04-22T06:32:05.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.657+0000 7f118a57e640 1 -- 172.21.15.44:0/3696302619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f118415d180 msgr2=0x7f11840cecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:05.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.657+0000 7f118a57e640 1 --2- 172.21.15.44:0/3696302619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f118415d180 0x7f11840cecd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f117c009e50 tx=0x7f117c02bff0 comp rx=0 tx=0).stop 2024-04-22T06:32:05.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.657+0000 7f118a57e640 1 -- 172.21.15.44:0/3696302619 shutdown_connections 2024-04-22T06:32:05.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.657+0000 7f118a57e640 1 --2- 172.21.15.44:0/3696302619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f118415d180 0x7f11840cecd0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:05.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.657+0000 7f118a57e640 1 --2- 172.21.15.44:0/3696302619 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11841551e0 0x7f11840ce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:05.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.657+0000 7f118a57e640 1 -- 172.21.15.44:0/3696302619 >> 172.21.15.44:0/3696302619 conn(0x7f118406fad0 msgr2=0x7f1184074ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:05.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.657+0000 7f118a57e640 1 -- 172.21.15.44:0/3696302619 shutdown_connections 2024-04-22T06:32:05.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.657+0000 7f118a57e640 1 -- 172.21.15.44:0/3696302619 wait complete. 2024-04-22T06:32:05.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.658+0000 7f118a57e640 1 Processor -- start 2024-04-22T06:32:05.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.658+0000 7f118a57e640 1 -- start start 2024-04-22T06:32:05.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.659+0000 7f118a57e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11841551e0 0x7f11841f20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:05.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.659+0000 7f118a57e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f118415d180 0x7f11840c65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:05.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.659+0000 7f118a57e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11841f16d0 0x7f1184153850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:05.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.659+0000 7f118a57e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f118415be60 con 0x7f11841551e0 2024-04-22T06:32:05.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.659+0000 7f118a57e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f118415bb80 con 0x7f11841f16d0 2024-04-22T06:32:05.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.659+0000 7f118a57e640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f118415c550 con 0x7f118415d180 2024-04-22T06:32:05.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.659+0000 7f1183fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11841551e0 0x7f11841f20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:05.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.659+0000 7f1183fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11841551e0 0x7f11841f20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52376/0 (socket says 172.21.15.44:52376) 2024-04-22T06:32:05.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.659+0000 7f1183fff640 1 -- 172.21.15.44:0/1677969934 learned_addr learned my addr 172.21.15.44:0/1677969934 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:05.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.659+0000 7f11837fe640 1 --2- 172.21.15.44:0/1677969934 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f118415d180 0x7f11840c65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:05.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.659+0000 7f1188b1b640 1 --2- 172.21.15.44:0/1677969934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11841f16d0 0x7f1184153850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:05.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.659+0000 7f1183fff640 1 -- 172.21.15.44:0/1677969934 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f118415d180 msgr2=0x7f11840c65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:05.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.659+0000 7f1183fff640 1 --2- 172.21.15.44:0/1677969934 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f118415d180 0x7f11840c65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:05.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.659+0000 7f1183fff640 1 -- 172.21.15.44:0/1677969934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11841f16d0 msgr2=0x7f1184153850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:05.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.659+0000 7f1183fff640 1 --2- 172.21.15.44:0/1677969934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11841f16d0 0x7f1184153850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:05.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.659+0000 7f1183fff640 1 -- 172.21.15.44:0/1677969934 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11841589e0 con 0x7f11841551e0 2024-04-22T06:32:05.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.659+0000 7f11837fe640 1 --2- 172.21.15.44:0/1677969934 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f118415d180 0x7f11840c65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:32:05.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.659+0000 7f1188b1b640 1 --2- 172.21.15.44:0/1677969934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11841f16d0 0x7f1184153850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:32:05.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.659+0000 7f1183fff640 1 --2- 172.21.15.44:0/1677969934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11841551e0 0x7f11841f20c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f117800d800 tx=0x7f117800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:05.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.659+0000 7f1180ff9640 1 -- 172.21.15.44:0/1677969934 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117800b6c0 con 0x7f11841551e0 2024-04-22T06:32:05.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.660+0000 7f1180ff9640 1 -- 172.21.15.44:0/1677969934 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1178019070 con 0x7f11841551e0 2024-04-22T06:32:05.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.660+0000 7f118a57e640 1 -- 172.21.15.44:0/1677969934 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11840c5b30 con 0x7f11841551e0 2024-04-22T06:32:05.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.660+0000 7f1180ff9640 1 -- 172.21.15.44:0/1677969934 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117800ba80 con 0x7f11841551e0 2024-04-22T06:32:05.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.660+0000 7f118a57e640 1 -- 172.21.15.44:0/1677969934 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11840c7a10 con 0x7f11841551e0 2024-04-22T06:32:05.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.661+0000 7f118a57e640 1 -- 172.21.15.44:0/1677969934 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f118406cd20 con 0x7f11841551e0 2024-04-22T06:32:05.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.661+0000 7f1180ff9640 1 -- 172.21.15.44:0/1677969934 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f117800bc20 con 0x7f11841551e0 2024-04-22T06:32:05.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.661+0000 7f1180ff9640 1 --2- 172.21.15.44:0/1677969934 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f117006c8f0 0x7f117006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:05.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.661+0000 7f1180ff9640 1 -- 172.21.15.44:0/1677969934 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f117808c500 con 0x7f11841551e0 2024-04-22T06:32:05.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.662+0000 7f11837fe640 1 --2- 172.21.15.44:0/1677969934 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f117006c8f0 0x7f117006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:05.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.662+0000 7f11837fe640 1 --2- 172.21.15.44:0/1677969934 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f117006c8f0 0x7f117006edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1174007fd0 tx=0x7f1174009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:05.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.666+0000 7f1180ff9640 1 -- 172.21.15.44:0/1677969934 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f118406cd20 con 0x7f11841551e0 2024-04-22T06:32:05.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.851+0000 7f118a57e640 1 -- 172.21.15.44:0/1677969934 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f118406d5c0 con 0x7f11841551e0 2024-04-22T06:32:05.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.852+0000 7f1180ff9640 1 -- 172.21.15.44:0/1677969934 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f118406d5c0 con 0x7f11841551e0 2024-04-22T06:32:05.851 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-04-22T06:32:05.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.854+0000 7f118a57e640 1 -- 172.21.15.44:0/1677969934 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f117006c8f0 msgr2=0x7f117006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:05.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.854+0000 7f118a57e640 1 --2- 172.21.15.44:0/1677969934 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f117006c8f0 0x7f117006edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1174007fd0 tx=0x7f1174009040 comp rx=0 tx=0).stop 2024-04-22T06:32:05.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.854+0000 7f118a57e640 1 -- 172.21.15.44:0/1677969934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11841551e0 msgr2=0x7f11841f20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:05.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.854+0000 7f118a57e640 1 --2- 172.21.15.44:0/1677969934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11841551e0 0x7f11841f20c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f117800d800 tx=0x7f117800dcc0 comp rx=0 tx=0).stop 2024-04-22T06:32:05.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.854+0000 7f118a57e640 1 -- 172.21.15.44:0/1677969934 shutdown_connections 2024-04-22T06:32:05.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.854+0000 7f118a57e640 1 --2- 172.21.15.44:0/1677969934 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f117006c8f0 0x7f117006edb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:05.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.854+0000 7f118a57e640 1 --2- 172.21.15.44:0/1677969934 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f118415d180 0x7f11840c65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:05.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.854+0000 7f118a57e640 1 --2- 172.21.15.44:0/1677969934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11841f16d0 0x7f1184153850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:05.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.854+0000 7f118a57e640 1 --2- 172.21.15.44:0/1677969934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11841551e0 0x7f11841f20c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:05.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.854+0000 7f118a57e640 1 -- 172.21.15.44:0/1677969934 >> 172.21.15.44:0/1677969934 conn(0x7f118406fad0 msgr2=0x7f1184159ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:05.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.854+0000 7f118a57e640 1 -- 172.21.15.44:0/1677969934 shutdown_connections 2024-04-22T06:32:05.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.854+0000 7f118a57e640 1 -- 172.21.15.44:0/1677969934 wait complete. 2024-04-22T06:32:05.864 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T06:32:05.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.965+0000 7fd440346640 1 Processor -- start 2024-04-22T06:32:05.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.965+0000 7fd440346640 1 -- start start 2024-04-22T06:32:05.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.966+0000 7fd440346640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd438155680 0x7fd43815bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:05.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.966+0000 7fd440346640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd4380cc4e0 0x7fd4380ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:05.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.966+0000 7fd440346640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd43806c6e0 con 0x7fd4381552a0 2024-04-22T06:32:05.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.966+0000 7fd440346640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fd4380cb990 con 0x7fd438155680 2024-04-22T06:32:05.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.966+0000 7fd440346640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fd43806c9c0 con 0x7fd4380cc4e0 2024-04-22T06:32:05.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.966+0000 7fd43d8e1640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd438155680 0x7fd43815bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:05.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.966+0000 7fd43e0e2640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fd4381552a0 0x7fd4380cbdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40130/0 (socket says 172.21.15.44:40130) 2024-04-22T06:32:05.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.966+0000 7fd43e0e2640 1 -- 172.21.15.44:0/1765516774 learned_addr learned my addr 172.21.15.44:0/1765516774 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:32:05.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.966+0000 7fd43e8e3640 1 --2- 172.21.15.44:0/1765516774 >> v2:172.21.15.70:3300/0 conn(0x7fd4380cc4e0 0x7fd4380ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:05.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.966+0000 7fd43d8e1640 1 -- 172.21.15.44:0/1765516774 >> v2:172.21.15.70:3300/0 conn(0x7fd4380cc4e0 msgr2=0x7fd4380ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:05.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.966+0000 7fd43d8e1640 1 --2- 172.21.15.44:0/1765516774 >> v2:172.21.15.70:3300/0 conn(0x7fd4380cc4e0 0x7fd4380ccfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:05.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.966+0000 7fd43d8e1640 1 -- 172.21.15.44:0/1765516774 >> v1:172.21.15.44:6789/0 conn(0x7fd4381552a0 legacy=0x7fd4380cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:05.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.966+0000 7fd43d0e0640 1 -- 172.21.15.44:0/1765516774 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3880557962 0 0) 0x7fd43806c6e0 con 0x7fd4381552a0 2024-04-22T06:32:05.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.966+0000 7fd43d8e1640 1 -- 172.21.15.44:0/1765516774 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd43806cca0 con 0x7fd438155680 2024-04-22T06:32:05.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.967+0000 7fd43d8e1640 1 --2- 172.21.15.44:0/1765516774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd438155680 0x7fd43815bb80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd438073f70 tx=0x7fd42802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eca13c2ee883941 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:05.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.967+0000 7fd43d0e0640 1 -- 172.21.15.44:0/1765516774 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd428034070 con 0x7fd438155680 2024-04-22T06:32:05.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.967+0000 7fd43d0e0640 1 -- 172.21.15.44:0/1765516774 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd428030610 con 0x7fd438155680 2024-04-22T06:32:05.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.967+0000 7fd43d0e0640 1 -- 172.21.15.44:0/1765516774 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4280309e0 con 0x7fd438155680 2024-04-22T06:32:05.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.968+0000 7fd440346640 1 -- 172.21.15.44:0/1765516774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd438155680 msgr2=0x7fd43815bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:05.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.968+0000 7fd440346640 1 --2- 172.21.15.44:0/1765516774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd438155680 0x7fd43815bb80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd438073f70 tx=0x7fd42802b960 comp rx=0 tx=0).stop 2024-04-22T06:32:05.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.968+0000 7fd440346640 1 -- 172.21.15.44:0/1765516774 shutdown_connections 2024-04-22T06:32:05.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.968+0000 7fd440346640 1 --2- 172.21.15.44:0/1765516774 >> v2:172.21.15.70:3300/0 conn(0x7fd4380cc4e0 0x7fd4380ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:05.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.968+0000 7fd440346640 1 --2- 172.21.15.44:0/1765516774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd438155680 0x7fd43815bb80 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:05.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.968+0000 7fd440346640 1 -- 172.21.15.44:0/1765516774 >> 172.21.15.44:0/1765516774 conn(0x7fd438075820 msgr2=0x7fd438073000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:05.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.968+0000 7fd440346640 1 -- 172.21.15.44:0/1765516774 shutdown_connections 2024-04-22T06:32:05.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.968+0000 7fd440346640 1 -- 172.21.15.44:0/1765516774 wait complete. 2024-04-22T06:32:05.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.969+0000 7fd440346640 1 Processor -- start 2024-04-22T06:32:05.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.969+0000 7fd440346640 1 -- start start 2024-04-22T06:32:05.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.969+0000 7fd440346640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4380cc4e0 0x7fd43814cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:05.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.969+0000 7fd440346640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4381552a0 0x7fd43815eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:05.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.969+0000 7fd440346640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd438150690 0x7fd43815efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:05.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.969+0000 7fd440346640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd43806d540 con 0x7fd4381552a0 2024-04-22T06:32:05.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.969+0000 7fd440346640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd43806d260 con 0x7fd4380cc4e0 2024-04-22T06:32:05.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.969+0000 7fd440346640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd4380b6690 con 0x7fd438150690 2024-04-22T06:32:05.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.970+0000 7fd43d8e1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4381552a0 0x7fd43815eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:05.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.970+0000 7fd43d8e1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4381552a0 0x7fd43815eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52392/0 (socket says 172.21.15.44:52392) 2024-04-22T06:32:05.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.970+0000 7fd43d8e1640 1 -- 172.21.15.44:0/2345528224 learned_addr learned my addr 172.21.15.44:0/2345528224 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:05.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.970+0000 7fd43e8e3640 1 --2- 172.21.15.44:0/2345528224 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd438150690 0x7fd43815efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:05.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.970+0000 7fd43e0e2640 1 --2- 172.21.15.44:0/2345528224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4380cc4e0 0x7fd43814cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:05.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.970+0000 7fd43d8e1640 1 -- 172.21.15.44:0/2345528224 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd438150690 msgr2=0x7fd43815efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:05.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.970+0000 7fd43d8e1640 1 --2- 172.21.15.44:0/2345528224 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd438150690 0x7fd43815efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:05.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.970+0000 7fd43d8e1640 1 -- 172.21.15.44:0/2345528224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4380cc4e0 msgr2=0x7fd43814cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:05.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.970+0000 7fd43d8e1640 1 --2- 172.21.15.44:0/2345528224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4380cc4e0 0x7fd43814cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:05.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.970+0000 7fd43d8e1640 1 -- 172.21.15.44:0/2345528224 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4381f1190 con 0x7fd4381552a0 2024-04-22T06:32:05.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.970+0000 7fd43e0e2640 1 --2- 172.21.15.44:0/2345528224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4380cc4e0 0x7fd43814cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:32:05.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.970+0000 7fd43d8e1640 1 --2- 172.21.15.44:0/2345528224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4381552a0 0x7fd43815eaa0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd42802bf70 tx=0x7fd4280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:05.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.970+0000 7fd42effd640 1 -- 172.21.15.44:0/2345528224 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd428034040 con 0x7fd4381552a0 2024-04-22T06:32:05.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.970+0000 7fd42effd640 1 -- 172.21.15.44:0/2345528224 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd42803aaa0 con 0x7fd4381552a0 2024-04-22T06:32:05.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.970+0000 7fd440346640 1 -- 172.21.15.44:0/2345528224 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4381f1450 con 0x7fd4381552a0 2024-04-22T06:32:05.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.970+0000 7fd42effd640 1 -- 172.21.15.44:0/2345528224 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4280443f0 con 0x7fd4381552a0 2024-04-22T06:32:05.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.970+0000 7fd440346640 1 -- 172.21.15.44:0/2345528224 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd43814d440 con 0x7fd4381552a0 2024-04-22T06:32:05.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.971+0000 7fd440346640 1 -- 172.21.15.44:0/2345528224 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd438068cc0 con 0x7fd4381552a0 2024-04-22T06:32:05.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.972+0000 7fd42effd640 1 -- 172.21.15.44:0/2345528224 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd428037070 con 0x7fd4381552a0 2024-04-22T06:32:05.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.972+0000 7fd42effd640 1 --2- 172.21.15.44:0/2345528224 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd41c06c8f0 0x7fd41c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:05.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.972+0000 7fd42effd640 1 -- 172.21.15.44:0/2345528224 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fd4280add00 con 0x7fd4381552a0 2024-04-22T06:32:05.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.972+0000 7fd43e0e2640 1 --2- 172.21.15.44:0/2345528224 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd41c06c8f0 0x7fd41c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:05.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.973+0000 7fd43e0e2640 1 --2- 172.21.15.44:0/2345528224 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd41c06c8f0 0x7fd41c06edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd434000f60 tx=0x7fd434009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:05.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:05.976+0000 7fd42effd640 1 -- 172.21.15.44:0/2345528224 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd42803f070 con 0x7fd4381552a0 2024-04-22T06:32:06.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.165+0000 7fd440346640 1 -- 172.21.15.44:0/2345528224 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4380c9e20 con 0x7fd4381552a0 2024-04-22T06:32:06.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.166+0000 7fd42effd640 1 -- 172.21.15.44:0/2345528224 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18794 (secure 0 0 0) 0x7fd42807bdc0 con 0x7fd4381552a0 2024-04-22T06:32:06.165 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:32:06.165 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":27,"fsid":"9b0a34ca-403d-4513-8a66-d3749ba559aa","created":"2024-04-22T06:30:42.334581+0000","modified":"2024-04-22T06:31:48.138913+0000","last_up_change":"2024-04-22T06:30:52.942257+0000","last_in_change":"2024-04-22T06:30:44.467727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T06:30:54.560256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T06:31:04.612387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T06:31:05.312124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-22T06:31:09.025550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b4a6692-53c9-472d-b2f3-1dd29a6679ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6817","nonce":2220268493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6819","nonce":2220268493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6823","nonce":2220268493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6821","nonce":2220268493}]},"public_addr":"172.21.15.44:6817/2220268493","cluster_addr":"172.21.15.44:6819/2220268493","heartbeat_back_addr":"172.21.15.44:6823/2220268493","heartbeat_front_addr":"172.21.15.44:6821/2220268493","state":["exists","up"]},{"osd":1,"uuid":"70a6370a-7c0a-4d64-a0e4-b2e726b4f291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6809","nonce":1278678977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6811","nonce":1278678977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6815","nonce":1278678977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6813","nonce":1278678977}]},"public_addr":"172.21.15.44:6809/1278678977","cluster_addr":"172.21.15.44:6811/1278678977","heartbeat_back_addr":"172.21.15.44:6815/1278678977","heartbeat_front_addr":"172.21.15.44:6813/1278678977","state":["exists","up"]},{"osd":2,"uuid":"f2e51a62-ca91-470d-bd5a-5710c61deee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6801","nonce":2544087903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6803","nonce":2544087903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6807","nonce":2544087903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6805","nonce":2544087903}]},"public_addr":"172.21.15.44:6801/2544087903","cluster_addr":"172.21.15.44:6803/2544087903","heartbeat_back_addr":"172.21.15.44:6807/2544087903","heartbeat_front_addr":"172.21.15.44:6805/2544087903","state":["exists","up"]},{"osd":3,"uuid":"bc514203-9af9-4f1f-b18e-765bdd5de6e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6825","nonce":1692928109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6827","nonce":1692928109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6831","nonce":1692928109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6829","nonce":1692928109}]},"public_addr":"172.21.15.44:6825/1692928109","cluster_addr":"172.21.15.44:6827/1692928109","heartbeat_back_addr":"172.21.15.44:6831/1692928109","heartbeat_front_addr":"172.21.15.44:6829/1692928109","state":["exists","up"]},{"osd":4,"uuid":"357c439a-6e08-4c19-ac18-b68e6c890497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6820","nonce":3802577427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6824","nonce":3802577427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6831","nonce":3802577427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6828","nonce":3802577427}]},"public_addr":"172.21.15.70:6820/3802577427","cluster_addr":"172.21.15.70:6824/3802577427","heartbeat_back_addr":"172.21.15.70:6831/3802577427","heartbeat_front_addr":"172.21.15.70:6828/3802577427","state":["exists","up"]},{"osd":5,"uuid":"952fb88f-d56e-4305-bfa8-d32674332dfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6801","nonce":664757345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6803","nonce":664757345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6807","nonce":664757345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6805","nonce":664757345}]},"public_addr":"172.21.15.70:6801/664757345","cluster_addr":"172.21.15.70:6803/664757345","heartbeat_back_addr":"172.21.15.70:6807/664757345","heartbeat_front_addr":"172.21.15.70:6805/664757345","state":["exists","up"]},{"osd":6,"uuid":"74399b3d-fac2-45ab-9172-2cd95b02450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6817","nonce":1623253482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6821","nonce":1623253482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6827","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6829","nonce":1623253482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6825","nonce":1623253482}]},"public_addr":"172.21.15.70:6817/1623253482","cluster_addr":"172.21.15.70:6821/1623253482","heartbeat_back_addr":"172.21.15.70:6829/1623253482","heartbeat_front_addr":"172.21.15.70:6825/1623253482","state":["exists","up"]},{"osd":7,"uuid":"8f35508d-6fcc-43ff-8ee7-0498a7ab3b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6809","nonce":1551180300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6811","nonce":1551180300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6815","nonce":1551180300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6813","nonce":1551180300}]},"public_addr":"172.21.15.70:6809/1551180300","cluster_addr":"172.21.15.70:6811/1551180300","heartbeat_back_addr":"172.21.15.70:6815/1551180300","heartbeat_front_addr":"172.21.15.70:6813/1551180300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.612502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.653756+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.521169+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.649955+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.640231+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.686638+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.642199+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.700360+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T06:32:06.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.168+0000 7fd440346640 1 -- 172.21.15.44:0/2345528224 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd41c06c8f0 msgr2=0x7fd41c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:06.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.168+0000 7fd440346640 1 --2- 172.21.15.44:0/2345528224 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd41c06c8f0 0x7fd41c06edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd434000f60 tx=0x7fd434009040 comp rx=0 tx=0).stop 2024-04-22T06:32:06.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.168+0000 7fd440346640 1 -- 172.21.15.44:0/2345528224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4381552a0 msgr2=0x7fd43815eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:06.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.168+0000 7fd440346640 1 --2- 172.21.15.44:0/2345528224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4381552a0 0x7fd43815eaa0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd42802bf70 tx=0x7fd4280046c0 comp rx=0 tx=0).stop 2024-04-22T06:32:06.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.168+0000 7fd440346640 1 -- 172.21.15.44:0/2345528224 shutdown_connections 2024-04-22T06:32:06.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.168+0000 7fd440346640 1 --2- 172.21.15.44:0/2345528224 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd41c06c8f0 0x7fd41c06edb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:06.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.168+0000 7fd440346640 1 --2- 172.21.15.44:0/2345528224 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd438150690 0x7fd43815efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:06.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.168+0000 7fd440346640 1 --2- 172.21.15.44:0/2345528224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4381552a0 0x7fd43815eaa0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:06.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.168+0000 7fd440346640 1 --2- 172.21.15.44:0/2345528224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4380cc4e0 0x7fd43814cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:06.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.168+0000 7fd440346640 1 -- 172.21.15.44:0/2345528224 >> 172.21.15.44:0/2345528224 conn(0x7fd438075820 msgr2=0x7fd438152d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:06.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.168+0000 7fd440346640 1 -- 172.21.15.44:0/2345528224 shutdown_connections 2024-04-22T06:32:06.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.168+0000 7fd440346640 1 -- 172.21.15.44:0/2345528224 wait complete. 2024-04-22T06:32:06.178 INFO:tasks.quiescer.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T06:30:54.560256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T06:31:04.612387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T06:31:05.312124+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-22T06:31:09.025550+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-22T06:32:06.178 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T06:32:06.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.283+0000 7f51f6806640 1 Processor -- start 2024-04-22T06:32:06.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.283+0000 7f51f6806640 1 -- start start 2024-04-22T06:32:06.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.284+0000 7f51f6806640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f51f015bc90 0x7f51f0070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:06.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.284+0000 7f51f6806640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f51f00711e0 0x7f51f0165b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:06.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.284+0000 7f51f6806640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51f006d4e0 con 0x7f51f01551e0 2024-04-22T06:32:06.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.284+0000 7f51f6806640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f51f0070810 con 0x7f51f00711e0 2024-04-22T06:32:06.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.284+0000 7f51f6806640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f51f0073380 con 0x7f51f015bc90 2024-04-22T06:32:06.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.284+0000 7f51f4da3640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f51f00711e0 0x7f51f0165b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:06.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.284+0000 7f51effff640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f51f01551e0 0x7f51f006b930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40138/0 (socket says 172.21.15.44:40138) 2024-04-22T06:32:06.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.284+0000 7f51effff640 1 -- 172.21.15.44:0/2103193435 learned_addr learned my addr 172.21.15.44:0/2103193435 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:32:06.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.285+0000 7f51ef7fe640 1 --2- 172.21.15.44:0/2103193435 >> v2:172.21.15.70:3300/0 conn(0x7f51f015bc90 0x7f51f0070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:06.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.285+0000 7f51f4da3640 1 -- 172.21.15.44:0/2103193435 >> v2:172.21.15.70:3300/0 conn(0x7f51f015bc90 msgr2=0x7f51f0070ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:06.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.285+0000 7f51f4da3640 1 --2- 172.21.15.44:0/2103193435 >> v2:172.21.15.70:3300/0 conn(0x7f51f015bc90 0x7f51f0070ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:06.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.285+0000 7f51f4da3640 1 -- 172.21.15.44:0/2103193435 >> v1:172.21.15.44:6789/0 conn(0x7f51f01551e0 legacy=0x7f51f006b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:06.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.285+0000 7f51f4da3640 1 -- 172.21.15.44:0/2103193435 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51f0073660 con 0x7f51f00711e0 2024-04-22T06:32:06.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.285+0000 7f51eeffd640 1 -- 172.21.15.44:0/2103193435 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 72480260 0 0) 0x7f51f006d4e0 con 0x7f51f01551e0 2024-04-22T06:32:06.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.286+0000 7f51f4da3640 1 --2- 172.21.15.44:0/2103193435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51f00711e0 0x7f51f0165b90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f51f0050d80 tx=0x7f51e802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e398ff5114f52a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:06.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.286+0000 7f51eeffd640 1 -- 172.21.15.44:0/2103193435 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e8039070 con 0x7f51f00711e0 2024-04-22T06:32:06.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.286+0000 7f51eeffd640 1 -- 172.21.15.44:0/2103193435 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51e802cc80 con 0x7f51f00711e0 2024-04-22T06:32:06.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.286+0000 7f51eeffd640 1 -- 172.21.15.44:0/2103193435 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e8034660 con 0x7f51f00711e0 2024-04-22T06:32:06.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.286+0000 7f51f6806640 1 -- 172.21.15.44:0/2103193435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51f00711e0 msgr2=0x7f51f0165b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:06.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.286+0000 7f51f6806640 1 --2- 172.21.15.44:0/2103193435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51f00711e0 0x7f51f0165b90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f51f0050d80 tx=0x7f51e802bed0 comp rx=0 tx=0).stop 2024-04-22T06:32:06.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.286+0000 7f51f6806640 1 -- 172.21.15.44:0/2103193435 shutdown_connections 2024-04-22T06:32:06.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.286+0000 7f51f6806640 1 --2- 172.21.15.44:0/2103193435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51f00711e0 0x7f51f0165b90 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:06.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.286+0000 7f51f6806640 1 --2- 172.21.15.44:0/2103193435 >> v2:172.21.15.70:3300/0 conn(0x7f51f015bc90 0x7f51f0070ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:06.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.286+0000 7f51f6806640 1 -- 172.21.15.44:0/2103193435 >> 172.21.15.44:0/2103193435 conn(0x7f51f00ce700 msgr2=0x7f51f00ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:06.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.286+0000 7f51f6806640 1 -- 172.21.15.44:0/2103193435 shutdown_connections 2024-04-22T06:32:06.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.286+0000 7f51f6806640 1 -- 172.21.15.44:0/2103193435 wait complete. 2024-04-22T06:32:06.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.287+0000 7f51f6806640 1 Processor -- start 2024-04-22T06:32:06.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.288+0000 7f51f6806640 1 -- start start 2024-04-22T06:32:06.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.288+0000 7f51f6806640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51f00711e0 0x7f51f015efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:06.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.288+0000 7f51f6806640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51f01551e0 0x7f51f01f1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:06.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.288+0000 7f51f6806640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51f015bc90 0x7f51f01f2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:06.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.288+0000 7f51f6806640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f51f00cf420 con 0x7f51f00711e0 2024-04-22T06:32:06.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.288+0000 7f51f6806640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f51f0073c20 con 0x7f51f01551e0 2024-04-22T06:32:06.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.288+0000 7f51f6806640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f51f00715f0 con 0x7f51f015bc90 2024-04-22T06:32:06.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.288+0000 7f51effff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51f00711e0 0x7f51f015efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:06.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.288+0000 7f51effff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51f00711e0 0x7f51f015efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52408/0 (socket says 172.21.15.44:52408) 2024-04-22T06:32:06.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.288+0000 7f51effff640 1 -- 172.21.15.44:0/1095343333 learned_addr learned my addr 172.21.15.44:0/1095343333 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:06.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.289+0000 7f51ef7fe640 1 --2- 172.21.15.44:0/1095343333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51f01551e0 0x7f51f01f1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:06.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.289+0000 7f51f4da3640 1 --2- 172.21.15.44:0/1095343333 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51f015bc90 0x7f51f01f2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:06.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.289+0000 7f51effff640 1 -- 172.21.15.44:0/1095343333 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51f015bc90 msgr2=0x7f51f01f2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:06.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.289+0000 7f51effff640 1 --2- 172.21.15.44:0/1095343333 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51f015bc90 0x7f51f01f2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:06.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.289+0000 7f51effff640 1 -- 172.21.15.44:0/1095343333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51f01551e0 msgr2=0x7f51f01f1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:06.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.289+0000 7f51effff640 1 --2- 172.21.15.44:0/1095343333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51f01551e0 0x7f51f01f1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:06.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.289+0000 7f51effff640 1 -- 172.21.15.44:0/1095343333 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51f015ca10 con 0x7f51f00711e0 2024-04-22T06:32:06.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.289+0000 7f51effff640 1 --2- 172.21.15.44:0/1095343333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51f00711e0 0x7f51f015efd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f51d800c4d0 tx=0x7f51d800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:06.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.289+0000 7f51ecff9640 1 -- 172.21.15.44:0/1095343333 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d8018070 con 0x7f51f00711e0 2024-04-22T06:32:06.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.289+0000 7f51ecff9640 1 -- 172.21.15.44:0/1095343333 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51d8007500 con 0x7f51f00711e0 2024-04-22T06:32:06.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.289+0000 7f51f6806640 1 -- 172.21.15.44:0/1095343333 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51f015ccb0 con 0x7f51f00711e0 2024-04-22T06:32:06.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.289+0000 7f51ecff9640 1 -- 172.21.15.44:0/1095343333 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d8004a20 con 0x7f51f00711e0 2024-04-22T06:32:06.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.289+0000 7f51f6806640 1 -- 172.21.15.44:0/1095343333 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51f015d270 con 0x7f51f00711e0 2024-04-22T06:32:06.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.290+0000 7f51f6806640 1 -- 172.21.15.44:0/1095343333 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51f0069310 con 0x7f51f00711e0 2024-04-22T06:32:06.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.291+0000 7f51ecff9640 1 -- 172.21.15.44:0/1095343333 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f51d8004bc0 con 0x7f51f00711e0 2024-04-22T06:32:06.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.291+0000 7f51ecff9640 1 --2- 172.21.15.44:0/1095343333 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f51d006c8f0 0x7f51d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:06.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.291+0000 7f51ecff9640 1 -- 172.21.15.44:0/1095343333 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f51d808b250 con 0x7f51f00711e0 2024-04-22T06:32:06.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.291+0000 7f51ef7fe640 1 --2- 172.21.15.44:0/1095343333 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f51d006c8f0 0x7f51d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:06.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.292+0000 7f51ef7fe640 1 --2- 172.21.15.44:0/1095343333 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f51d006c8f0 0x7f51d006edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f51f00c8cb0 tx=0x7f51e0009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:06.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.296+0000 7f51ecff9640 1 -- 172.21.15.44:0/1095343333 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f51d8058b20 con 0x7f51f00711e0 2024-04-22T06:32:06.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.481+0000 7f51f6806640 1 -- 172.21.15.44:0/1095343333 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f51f006d120 con 0x7f51f00711e0 2024-04-22T06:32:06.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.481+0000 7f51ecff9640 1 -- 172.21.15.44:0/1095343333 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f51d805ce20 con 0x7f51f00711e0 2024-04-22T06:32:06.480 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-04-22T06:32:06.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.483+0000 7f51f6806640 1 -- 172.21.15.44:0/1095343333 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f51d006c8f0 msgr2=0x7f51d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:06.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.483+0000 7f51f6806640 1 --2- 172.21.15.44:0/1095343333 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f51d006c8f0 0x7f51d006edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f51f00c8cb0 tx=0x7f51e0009190 comp rx=0 tx=0).stop 2024-04-22T06:32:06.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.483+0000 7f51f6806640 1 -- 172.21.15.44:0/1095343333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51f00711e0 msgr2=0x7f51f015efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:06.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.483+0000 7f51f6806640 1 --2- 172.21.15.44:0/1095343333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51f00711e0 0x7f51f015efd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f51d800c4d0 tx=0x7f51d800c990 comp rx=0 tx=0).stop 2024-04-22T06:32:06.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.483+0000 7f51f6806640 1 -- 172.21.15.44:0/1095343333 shutdown_connections 2024-04-22T06:32:06.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.483+0000 7f51f6806640 1 --2- 172.21.15.44:0/1095343333 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f51d006c8f0 0x7f51d006edb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:06.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.483+0000 7f51f6806640 1 --2- 172.21.15.44:0/1095343333 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51f015bc90 0x7f51f01f2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:06.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.483+0000 7f51f6806640 1 --2- 172.21.15.44:0/1095343333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51f01551e0 0x7f51f01f1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:06.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.483+0000 7f51f6806640 1 --2- 172.21.15.44:0/1095343333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51f00711e0 0x7f51f015efd0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:06.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.483+0000 7f51f6806640 1 -- 172.21.15.44:0/1095343333 >> 172.21.15.44:0/1095343333 conn(0x7f51f00ce700 msgr2=0x7f51f006b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:06.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.483+0000 7f51f6806640 1 -- 172.21.15.44:0/1095343333 shutdown_connections 2024-04-22T06:32:06.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.483+0000 7f51f6806640 1 -- 172.21.15.44:0/1095343333 wait complete. 2024-04-22T06:32:06.493 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T06:32:06.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.598+0000 7fd49d86a640 1 Processor -- start 2024-04-22T06:32:06.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.598+0000 7fd49d86a640 1 -- start start 2024-04-22T06:32:06.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.599+0000 7fd49d86a640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd4981555c0 0x7fd49815ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:06.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.599+0000 7fd49d86a640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd49815cfb0 0x7fd498071170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:06.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.599+0000 7fd49d86a640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd49815b060 con 0x7fd4981551e0 2024-04-22T06:32:06.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.599+0000 7fd49d86a640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fd49815b300 con 0x7fd49815cfb0 2024-04-22T06:32:06.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.599+0000 7fd49d86a640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd49806c460 con 0x7fd4981555c0 2024-04-22T06:32:06.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.599+0000 7fd4977fe640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd49815cfb0 0x7fd498071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:06.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.599+0000 7fd496ffd640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fd4981551e0 0x7fd4980ce790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40148/0 (socket says 172.21.15.44:40148) 2024-04-22T06:32:06.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.599+0000 7fd496ffd640 1 -- 172.21.15.44:0/121997889 learned_addr learned my addr 172.21.15.44:0/121997889 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:32:06.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.599+0000 7fd48ffff640 1 --2- 172.21.15.44:0/121997889 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd4981555c0 0x7fd49815ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:06.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.599+0000 7fd4967fc640 1 -- 172.21.15.44:0/121997889 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4202520155 0 0) 0x7fd49815b060 con 0x7fd4981551e0 2024-04-22T06:32:06.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.599+0000 7fd4967fc640 1 -- 172.21.15.44:0/121997889 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd484003520 con 0x7fd4981551e0 2024-04-22T06:32:06.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.599+0000 7fd4977fe640 1 -- 172.21.15.44:0/121997889 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd4981555c0 msgr2=0x7fd49815ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:06.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.599+0000 7fd4977fe640 1 --2- 172.21.15.44:0/121997889 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd4981555c0 0x7fd49815ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:06.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.599+0000 7fd4977fe640 1 -- 172.21.15.44:0/121997889 >> v1:172.21.15.44:6789/0 conn(0x7fd4981551e0 legacy=0x7fd4980ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:06.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.599+0000 7fd4977fe640 1 -- 172.21.15.44:0/121997889 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd49815b5c0 con 0x7fd49815cfb0 2024-04-22T06:32:06.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.600+0000 7fd4977fe640 1 --2- 172.21.15.44:0/121997889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd49815cfb0 0x7fd498071170 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd490009d90 tx=0x7fd49002bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8a0e71af4ce2e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:06.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.600+0000 7fd4967fc640 1 -- 172.21.15.44:0/121997889 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd490039070 con 0x7fd49815cfb0 2024-04-22T06:32:06.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.600+0000 7fd4967fc640 1 -- 172.21.15.44:0/121997889 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd49002cce0 con 0x7fd49815cfb0 2024-04-22T06:32:06.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.600+0000 7fd4967fc640 1 -- 172.21.15.44:0/121997889 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd490034660 con 0x7fd49815cfb0 2024-04-22T06:32:06.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.601+0000 7fd49d86a640 1 -- 172.21.15.44:0/121997889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd49815cfb0 msgr2=0x7fd498071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:06.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.601+0000 7fd49d86a640 1 --2- 172.21.15.44:0/121997889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd49815cfb0 0x7fd498071170 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd490009d90 tx=0x7fd49002bf30 comp rx=0 tx=0).stop 2024-04-22T06:32:06.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.601+0000 7fd49d86a640 1 -- 172.21.15.44:0/121997889 shutdown_connections 2024-04-22T06:32:06.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.601+0000 7fd49d86a640 1 --2- 172.21.15.44:0/121997889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd49815cfb0 0x7fd498071170 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:06.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.601+0000 7fd49d86a640 1 --2- 172.21.15.44:0/121997889 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd4981555c0 0x7fd49815ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:06.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.601+0000 7fd49d86a640 1 -- 172.21.15.44:0/121997889 >> 172.21.15.44:0/121997889 conn(0x7fd49806fad0 msgr2=0x7fd498074ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:06.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.601+0000 7fd49d86a640 1 -- 172.21.15.44:0/121997889 shutdown_connections 2024-04-22T06:32:06.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.601+0000 7fd49d86a640 1 -- 172.21.15.44:0/121997889 wait complete. 2024-04-22T06:32:06.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.602+0000 7fd49d86a640 1 Processor -- start 2024-04-22T06:32:06.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.602+0000 7fd49d86a640 1 -- start start 2024-04-22T06:32:06.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.603+0000 7fd49d86a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4981551e0 0x7fd4980c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:06.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.603+0000 7fd49d86a640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd49815cfb0 0x7fd4980ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:06.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.603+0000 7fd49d86a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4980c5530 0x7fd4980cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:06.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.603+0000 7fd49d86a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd49815be60 con 0x7fd4980c5530 2024-04-22T06:32:06.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.603+0000 7fd49d86a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd49815bb80 con 0x7fd4981551e0 2024-04-22T06:32:06.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.603+0000 7fd49d86a640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd49815c550 con 0x7fd49815cfb0 2024-04-22T06:32:06.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.603+0000 7fd4977fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4980c5530 0x7fd4980cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:06.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.603+0000 7fd4977fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4980c5530 0x7fd4980cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52424/0 (socket says 172.21.15.44:52424) 2024-04-22T06:32:06.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.603+0000 7fd4977fe640 1 -- 172.21.15.44:0/3638577666 learned_addr learned my addr 172.21.15.44:0/3638577666 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:06.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.603+0000 7fd48ffff640 1 --2- 172.21.15.44:0/3638577666 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd49815cfb0 0x7fd4980ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:06.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.603+0000 7fd496ffd640 1 --2- 172.21.15.44:0/3638577666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4981551e0 0x7fd4980c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:06.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.603+0000 7fd4977fe640 1 -- 172.21.15.44:0/3638577666 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd49815cfb0 msgr2=0x7fd4980ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:06.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.603+0000 7fd4977fe640 1 --2- 172.21.15.44:0/3638577666 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd49815cfb0 0x7fd4980ca870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:06.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.603+0000 7fd4977fe640 1 -- 172.21.15.44:0/3638577666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4981551e0 msgr2=0x7fd4980c87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:06.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.603+0000 7fd4977fe640 1 --2- 172.21.15.44:0/3638577666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4981551e0 0x7fd4980c87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:06.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.603+0000 7fd4977fe640 1 -- 172.21.15.44:0/3638577666 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4981f43a0 con 0x7fd4980c5530 2024-04-22T06:32:06.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.603+0000 7fd4977fe640 1 --2- 172.21.15.44:0/3638577666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4980c5530 0x7fd4980cadb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd490009eb0 tx=0x7fd490004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:06.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.603+0000 7fd494ff9640 1 -- 172.21.15.44:0/3638577666 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd490039050 con 0x7fd4980c5530 2024-04-22T06:32:06.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.603+0000 7fd494ff9640 1 -- 172.21.15.44:0/3638577666 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd490045070 con 0x7fd4980c5530 2024-04-22T06:32:06.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.603+0000 7fd49d86a640 1 -- 172.21.15.44:0/3638577666 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4981f4640 con 0x7fd4980c5530 2024-04-22T06:32:06.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.603+0000 7fd494ff9640 1 -- 172.21.15.44:0/3638577666 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd490009450 con 0x7fd4980c5530 2024-04-22T06:32:06.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.603+0000 7fd49d86a640 1 -- 172.21.15.44:0/3638577666 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd498156f30 con 0x7fd4980c5530 2024-04-22T06:32:06.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.605+0000 7fd49d86a640 1 -- 172.21.15.44:0/3638577666 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd49806d5c0 con 0x7fd4980c5530 2024-04-22T06:32:06.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.605+0000 7fd494ff9640 1 -- 172.21.15.44:0/3638577666 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd49003e850 con 0x7fd4980c5530 2024-04-22T06:32:06.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.605+0000 7fd494ff9640 1 --2- 172.21.15.44:0/3638577666 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd47806c8f0 0x7fd47806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:06.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.605+0000 7fd494ff9640 1 -- 172.21.15.44:0/3638577666 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fd4900aeb80 con 0x7fd4980c5530 2024-04-22T06:32:06.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.605+0000 7fd496ffd640 1 --2- 172.21.15.44:0/3638577666 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd47806c8f0 0x7fd47806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:06.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.606+0000 7fd496ffd640 1 --2- 172.21.15.44:0/3638577666 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd47806c8f0 0x7fd47806edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd488000f60 tx=0x7fd488009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:06.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.609+0000 7fd494ff9640 1 -- 172.21.15.44:0/3638577666 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd49806d5c0 con 0x7fd4980c5530 2024-04-22T06:32:06.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.795+0000 7fd49d86a640 1 -- 172.21.15.44:0/3638577666 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd498068cc0 con 0x7fd4980c5530 2024-04-22T06:32:06.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.795+0000 7fd494ff9640 1 -- 172.21.15.44:0/3638577666 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fd490032e20 con 0x7fd4980c5530 2024-04-22T06:32:06.794 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-04-22T06:32:06.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.797+0000 7fd49d86a640 1 -- 172.21.15.44:0/3638577666 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd47806c8f0 msgr2=0x7fd47806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:06.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.797+0000 7fd49d86a640 1 --2- 172.21.15.44:0/3638577666 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd47806c8f0 0x7fd47806edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd488000f60 tx=0x7fd488009040 comp rx=0 tx=0).stop 2024-04-22T06:32:06.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.797+0000 7fd49d86a640 1 -- 172.21.15.44:0/3638577666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4980c5530 msgr2=0x7fd4980cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:06.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.797+0000 7fd49d86a640 1 --2- 172.21.15.44:0/3638577666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4980c5530 0x7fd4980cadb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd490009eb0 tx=0x7fd490004a40 comp rx=0 tx=0).stop 2024-04-22T06:32:06.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.797+0000 7fd49d86a640 1 -- 172.21.15.44:0/3638577666 shutdown_connections 2024-04-22T06:32:06.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.797+0000 7fd49d86a640 1 --2- 172.21.15.44:0/3638577666 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd47806c8f0 0x7fd47806edb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:06.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.797+0000 7fd49d86a640 1 --2- 172.21.15.44:0/3638577666 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd49815cfb0 0x7fd4980ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:06.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.797+0000 7fd49d86a640 1 --2- 172.21.15.44:0/3638577666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4980c5530 0x7fd4980cadb0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:06.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.797+0000 7fd49d86a640 1 --2- 172.21.15.44:0/3638577666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4981551e0 0x7fd4980c87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:06.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.797+0000 7fd49d86a640 1 -- 172.21.15.44:0/3638577666 >> 172.21.15.44:0/3638577666 conn(0x7fd49806fad0 msgr2=0x7fd498074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:06.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.797+0000 7fd49d86a640 1 -- 172.21.15.44:0/3638577666 shutdown_connections 2024-04-22T06:32:06.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.797+0000 7fd49d86a640 1 -- 172.21.15.44:0/3638577666 wait complete. 2024-04-22T06:32:06.807 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T06:32:06.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.911+0000 7fd7f1e81640 1 Processor -- start 2024-04-22T06:32:06.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.911+0000 7fd7f1e81640 1 -- start start 2024-04-22T06:32:06.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.912+0000 7fd7f1e81640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7ec151af0 0x7fd7ec0ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:06.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.912+0000 7fd7f1e81640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd7ec073ef0 0x7fd7ec0ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:06.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.912+0000 7fd7f1e81640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd7ec073910 0x7fd7ec15c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:06.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.912+0000 7fd7f1e81640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fd7ec15c0e0 con 0x7fd7ec073ef0 2024-04-22T06:32:06.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.912+0000 7fd7f1e81640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fd7ec15b4d0 con 0x7fd7ec073910 2024-04-22T06:32:06.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.912+0000 7fd7f1e81640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd7ec15b7b0 con 0x7fd7ec151af0 2024-04-22T06:32:06.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.912+0000 7fd7eaffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd7ec073ef0 0x7fd7ec0ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:06.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.912+0000 7fd7eaffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd7ec073ef0 0x7fd7ec0ced10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52428/0 (socket says 172.21.15.44:52428) 2024-04-22T06:32:06.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.912+0000 7fd7eaffd640 1 -- 172.21.15.44:0/1923307666 learned_addr learned my addr 172.21.15.44:0/1923307666 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:06.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.912+0000 7fd7eb7fe640 1 --2- 172.21.15.44:0/1923307666 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7ec151af0 0x7fd7ec0ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:06.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.912+0000 7fd7ebfff640 1 --2- 172.21.15.44:0/1923307666 >> v2:172.21.15.70:3300/0 conn(0x7fd7ec073910 0x7fd7ec15c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:06.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.913+0000 7fd7eaffd640 1 -- 172.21.15.44:0/1923307666 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7ec151af0 msgr2=0x7fd7ec0ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:06.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.913+0000 7fd7eaffd640 1 --2- 172.21.15.44:0/1923307666 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7ec151af0 0x7fd7ec0ce7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:06.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.913+0000 7fd7eaffd640 1 -- 172.21.15.44:0/1923307666 >> v2:172.21.15.70:3300/0 conn(0x7fd7ec073910 msgr2=0x7fd7ec15c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:06.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.913+0000 7fd7eaffd640 1 --2- 172.21.15.44:0/1923307666 >> v2:172.21.15.70:3300/0 conn(0x7fd7ec073910 0x7fd7ec15c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:06.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.913+0000 7fd7eaffd640 1 -- 172.21.15.44:0/1923307666 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7ec15ba90 con 0x7fd7ec073ef0 2024-04-22T06:32:06.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.913+0000 7fd7eaffd640 1 --2- 172.21.15.44:0/1923307666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7ec073ef0 0x7fd7ec0ced10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec073110 tx=0x7fd7e002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d8c98a87cccfc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:06.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.914+0000 7fd7ea7fc640 1 -- 172.21.15.44:0/1923307666 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e0035020 con 0x7fd7ec073ef0 2024-04-22T06:32:06.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.914+0000 7fd7ea7fc640 1 -- 172.21.15.44:0/1923307666 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7e0031870 con 0x7fd7ec073ef0 2024-04-22T06:32:06.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.914+0000 7fd7ea7fc640 1 -- 172.21.15.44:0/1923307666 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e0031c40 con 0x7fd7ec073ef0 2024-04-22T06:32:06.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.914+0000 7fd7f1e81640 1 -- 172.21.15.44:0/1923307666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7ec073ef0 msgr2=0x7fd7ec0ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:06.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.914+0000 7fd7f1e81640 1 --2- 172.21.15.44:0/1923307666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7ec073ef0 0x7fd7ec0ced10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec073110 tx=0x7fd7e002b910 comp rx=0 tx=0).stop 2024-04-22T06:32:06.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.914+0000 7fd7f1e81640 1 -- 172.21.15.44:0/1923307666 shutdown_connections 2024-04-22T06:32:06.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.914+0000 7fd7f1e81640 1 --2- 172.21.15.44:0/1923307666 >> v2:172.21.15.70:3300/0 conn(0x7fd7ec073910 0x7fd7ec15c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:06.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.914+0000 7fd7f1e81640 1 --2- 172.21.15.44:0/1923307666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7ec073ef0 0x7fd7ec0ced10 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:06.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.914+0000 7fd7f1e81640 1 --2- 172.21.15.44:0/1923307666 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7ec151af0 0x7fd7ec0ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:06.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.914+0000 7fd7f1e81640 1 -- 172.21.15.44:0/1923307666 >> 172.21.15.44:0/1923307666 conn(0x7fd7ec06f5d0 msgr2=0x7fd7ec070d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:06.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.914+0000 7fd7f1e81640 1 -- 172.21.15.44:0/1923307666 shutdown_connections 2024-04-22T06:32:06.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.914+0000 7fd7f1e81640 1 -- 172.21.15.44:0/1923307666 wait complete. 2024-04-22T06:32:06.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.915+0000 7fd7f1e81640 1 Processor -- start 2024-04-22T06:32:06.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.915+0000 7fd7f1e81640 1 -- start start 2024-04-22T06:32:06.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.916+0000 7fd7f1e81640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7ec073910 0x7fd7ec14cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:06.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.916+0000 7fd7f1e81640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7ec073ef0 0x7fd7ec1f1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:06.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.916+0000 7fd7f1e81640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7ec151af0 0x7fd7ec1431e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:06.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.916+0000 7fd7f1e81640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ec15b180 con 0x7fd7ec151af0 2024-04-22T06:32:06.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.916+0000 7fd7f1e81640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ec155790 con 0x7fd7ec073910 2024-04-22T06:32:06.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.916+0000 7fd7f1e81640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd7ec151ed0 con 0x7fd7ec073ef0 2024-04-22T06:32:06.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.916+0000 7fd7ebfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7ec151af0 0x7fd7ec1431e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:06.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.916+0000 7fd7ebfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7ec151af0 0x7fd7ec1431e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52438/0 (socket says 172.21.15.44:52438) 2024-04-22T06:32:06.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.916+0000 7fd7ebfff640 1 -- 172.21.15.44:0/3957667612 learned_addr learned my addr 172.21.15.44:0/3957667612 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:06.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.916+0000 7fd7eb7fe640 1 --2- 172.21.15.44:0/3957667612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7ec073910 0x7fd7ec14cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:06.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.916+0000 7fd7eaffd640 1 --2- 172.21.15.44:0/3957667612 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7ec073ef0 0x7fd7ec1f1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:06.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.917+0000 7fd7ebfff640 1 -- 172.21.15.44:0/3957667612 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7ec073ef0 msgr2=0x7fd7ec1f1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:06.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.917+0000 7fd7ebfff640 1 --2- 172.21.15.44:0/3957667612 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7ec073ef0 0x7fd7ec1f1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:06.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.917+0000 7fd7ebfff640 1 -- 172.21.15.44:0/3957667612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7ec073910 msgr2=0x7fd7ec14cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:06.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.917+0000 7fd7ebfff640 1 --2- 172.21.15.44:0/3957667612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7ec073910 0x7fd7ec14cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:06.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.917+0000 7fd7ebfff640 1 -- 172.21.15.44:0/3957667612 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7ec15f070 con 0x7fd7ec151af0 2024-04-22T06:32:06.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.917+0000 7fd7ebfff640 1 --2- 172.21.15.44:0/3957667612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7ec151af0 0x7fd7ec1431e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd7e400eed0 tx=0x7fd7e400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:06.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.917+0000 7fd7d3fff640 1 -- 172.21.15.44:0/3957667612 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e4018020 con 0x7fd7ec151af0 2024-04-22T06:32:06.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.917+0000 7fd7d3fff640 1 -- 172.21.15.44:0/3957667612 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7e400bd30 con 0x7fd7ec151af0 2024-04-22T06:32:06.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.917+0000 7fd7d3fff640 1 -- 172.21.15.44:0/3957667612 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e4014640 con 0x7fd7ec151af0 2024-04-22T06:32:06.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.917+0000 7fd7f1e81640 1 -- 172.21.15.44:0/3957667612 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7ec0c5da0 con 0x7fd7ec151af0 2024-04-22T06:32:06.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.917+0000 7fd7f1e81640 1 -- 172.21.15.44:0/3957667612 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7ec1493f0 con 0x7fd7ec151af0 2024-04-22T06:32:06.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.918+0000 7fd7f1e81640 1 -- 172.21.15.44:0/3957667612 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7ec069310 con 0x7fd7ec151af0 2024-04-22T06:32:06.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.918+0000 7fd7d3fff640 1 -- 172.21.15.44:0/3957667612 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd7e4005d10 con 0x7fd7ec151af0 2024-04-22T06:32:06.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.919+0000 7fd7d3fff640 1 --2- 172.21.15.44:0/3957667612 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd7d806c920 0x7fd7d806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:06.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.919+0000 7fd7d3fff640 1 -- 172.21.15.44:0/3957667612 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fd7e408c760 con 0x7fd7ec151af0 2024-04-22T06:32:06.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.919+0000 7fd7eb7fe640 1 --2- 172.21.15.44:0/3957667612 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd7d806c920 0x7fd7d806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:06.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.919+0000 7fd7eb7fe640 1 --2- 172.21.15.44:0/3957667612 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd7d806c920 0x7fd7d806ede0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc009ad0 tx=0x7fd7dc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:06.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:06.923+0000 7fd7d3fff640 1 -- 172.21.15.44:0/3957667612 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd7e4022030 con 0x7fd7ec151af0 2024-04-22T06:32:07.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.109+0000 7fd7f1e81640 1 -- 172.21.15.44:0/3957667612 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd7ec15aed0 con 0x7fd7ec151af0 2024-04-22T06:32:07.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.110+0000 7fd7d3fff640 1 -- 172.21.15.44:0/3957667612 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fd7e405a790 con 0x7fd7ec151af0 2024-04-22T06:32:07.109 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-04-22T06:32:07.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.112+0000 7fd7f1e81640 1 -- 172.21.15.44:0/3957667612 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd7d806c920 msgr2=0x7fd7d806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:07.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.112+0000 7fd7f1e81640 1 --2- 172.21.15.44:0/3957667612 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd7d806c920 0x7fd7d806ede0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc009ad0 tx=0x7fd7dc008040 comp rx=0 tx=0).stop 2024-04-22T06:32:07.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.112+0000 7fd7f1e81640 1 -- 172.21.15.44:0/3957667612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7ec151af0 msgr2=0x7fd7ec1431e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:07.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.112+0000 7fd7f1e81640 1 --2- 172.21.15.44:0/3957667612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7ec151af0 0x7fd7ec1431e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd7e400eed0 tx=0x7fd7e400b480 comp rx=0 tx=0).stop 2024-04-22T06:32:07.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.112+0000 7fd7f1e81640 1 -- 172.21.15.44:0/3957667612 shutdown_connections 2024-04-22T06:32:07.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.112+0000 7fd7f1e81640 1 --2- 172.21.15.44:0/3957667612 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd7d806c920 0x7fd7d806ede0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:07.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.112+0000 7fd7f1e81640 1 --2- 172.21.15.44:0/3957667612 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd7ec073ef0 0x7fd7ec1f1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:07.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.112+0000 7fd7f1e81640 1 --2- 172.21.15.44:0/3957667612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7ec151af0 0x7fd7ec1431e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:07.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.112+0000 7fd7f1e81640 1 --2- 172.21.15.44:0/3957667612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7ec073910 0x7fd7ec14cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:07.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.112+0000 7fd7f1e81640 1 -- 172.21.15.44:0/3957667612 >> 172.21.15.44:0/3957667612 conn(0x7fd7ec06f5d0 msgr2=0x7fd7ec070d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:07.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.112+0000 7fd7f1e81640 1 -- 172.21.15.44:0/3957667612 shutdown_connections 2024-04-22T06:32:07.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.112+0000 7fd7f1e81640 1 -- 172.21.15.44:0/3957667612 wait complete. 2024-04-22T06:32:07.121 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-22T06:32:07.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.229+0000 7f376e648640 1 Processor -- start 2024-04-22T06:32:07.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.229+0000 7f376e648640 1 -- start start 2024-04-22T06:32:07.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.230+0000 7f376e648640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37681551e0 0x7f376815bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:07.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.230+0000 7f376e648640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f376806b930 0x7f376806bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:07.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.230+0000 7f376e648640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f376806d4e0 con 0x7f3768070ca0 2024-04-22T06:32:07.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.230+0000 7f376e648640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f3768070810 con 0x7f376806b930 2024-04-22T06:32:07.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.230+0000 7f376e648640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3768073380 con 0x7f37681551e0 2024-04-22T06:32:07.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.230+0000 7f37677fe640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f376806b930 0x7f376806bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:07.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.230+0000 7f376cbe5640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f3768070ca0 0x7f37680710f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40156/0 (socket says 172.21.15.44:40156) 2024-04-22T06:32:07.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.230+0000 7f376cbe5640 1 -- 172.21.15.44:0/1560372815 learned_addr learned my addr 172.21.15.44:0/1560372815 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:32:07.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.231+0000 7f3767fff640 1 --2- 172.21.15.44:0/1560372815 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37681551e0 0x7f376815bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:07.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.231+0000 7f37677fe640 1 -- 172.21.15.44:0/1560372815 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37681551e0 msgr2=0x7f376815bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:07.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.231+0000 7f37677fe640 1 --2- 172.21.15.44:0/1560372815 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37681551e0 0x7f376815bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:07.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.231+0000 7f37677fe640 1 -- 172.21.15.44:0/1560372815 >> v1:172.21.15.44:6789/0 conn(0x7f3768070ca0 legacy=0x7f37680710f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:07.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.231+0000 7f37677fe640 1 -- 172.21.15.44:0/1560372815 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3768073660 con 0x7f376806b930 2024-04-22T06:32:07.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.231+0000 7f3766ffd640 1 -- 172.21.15.44:0/1560372815 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 737968894 0 0) 0x7f376806d4e0 con 0x7f3768070ca0 2024-04-22T06:32:07.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.231+0000 7f37677fe640 1 --2- 172.21.15.44:0/1560372815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f376806b930 0x7f376806bd70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f375c0097a0 tx=0x7f375c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad6ccd3ece7d504b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:07.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.232+0000 7f3766ffd640 1 -- 172.21.15.44:0/1560372815 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c035020 con 0x7f376806b930 2024-04-22T06:32:07.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.232+0000 7f3766ffd640 1 -- 172.21.15.44:0/1560372815 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f375c031830 con 0x7f376806b930 2024-04-22T06:32:07.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.232+0000 7f3766ffd640 1 -- 172.21.15.44:0/1560372815 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c031c00 con 0x7f376806b930 2024-04-22T06:32:07.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.232+0000 7f376e648640 1 -- 172.21.15.44:0/1560372815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f376806b930 msgr2=0x7f376806bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:07.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.232+0000 7f376e648640 1 --2- 172.21.15.44:0/1560372815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f376806b930 0x7f376806bd70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f375c0097a0 tx=0x7f375c02b8d0 comp rx=0 tx=0).stop 2024-04-22T06:32:07.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.232+0000 7f376e648640 1 -- 172.21.15.44:0/1560372815 shutdown_connections 2024-04-22T06:32:07.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.232+0000 7f376e648640 1 --2- 172.21.15.44:0/1560372815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f376806b930 0x7f376806bd70 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:07.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.232+0000 7f376e648640 1 --2- 172.21.15.44:0/1560372815 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37681551e0 0x7f376815bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:07.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.232+0000 7f376e648640 1 -- 172.21.15.44:0/1560372815 >> 172.21.15.44:0/1560372815 conn(0x7f37680ce700 msgr2=0x7f37680ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:07.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.232+0000 7f376e648640 1 -- 172.21.15.44:0/1560372815 shutdown_connections 2024-04-22T06:32:07.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.232+0000 7f376e648640 1 -- 172.21.15.44:0/1560372815 wait complete. 2024-04-22T06:32:07.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.233+0000 7f376e648640 1 Processor -- start 2024-04-22T06:32:07.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.234+0000 7f376e648640 1 -- start start 2024-04-22T06:32:07.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.234+0000 7f376e648640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f376806b930 0x7f37681f5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:07.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.234+0000 7f376e648640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3768070ca0 0x7f3768147500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:07.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.234+0000 7f376e648640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37681551e0 0x7f3768158030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:07.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.234+0000 7f376e648640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f37680cf420 con 0x7f3768070ca0 2024-04-22T06:32:07.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.234+0000 7f376e648640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f3768073c20 con 0x7f376806b930 2024-04-22T06:32:07.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.234+0000 7f376e648640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f37681555c0 con 0x7f37681551e0 2024-04-22T06:32:07.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.234+0000 7f37677fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3768070ca0 0x7f3768147500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:07.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.234+0000 7f37677fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3768070ca0 0x7f3768147500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52462/0 (socket says 172.21.15.44:52462) 2024-04-22T06:32:07.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.234+0000 7f37677fe640 1 -- 172.21.15.44:0/190726718 learned_addr learned my addr 172.21.15.44:0/190726718 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:07.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.234+0000 7f376cbe5640 1 --2- 172.21.15.44:0/190726718 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37681551e0 0x7f3768158030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:07.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.234+0000 7f3767fff640 1 --2- 172.21.15.44:0/190726718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f376806b930 0x7f37681f5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:07.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.235+0000 7f37677fe640 1 -- 172.21.15.44:0/190726718 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37681551e0 msgr2=0x7f3768158030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:07.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.235+0000 7f37677fe640 1 --2- 172.21.15.44:0/190726718 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37681551e0 0x7f3768158030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:07.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.235+0000 7f37677fe640 1 -- 172.21.15.44:0/190726718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f376806b930 msgr2=0x7f37681f5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:07.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.235+0000 7f37677fe640 1 --2- 172.21.15.44:0/190726718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f376806b930 0x7f37681f5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:07.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.235+0000 7f37677fe640 1 -- 172.21.15.44:0/190726718 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37681f1660 con 0x7f3768070ca0 2024-04-22T06:32:07.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.235+0000 7f37677fe640 1 --2- 172.21.15.44:0/190726718 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3768070ca0 0x7f3768147500 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f375c034040 tx=0x7f375c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:07.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.235+0000 7f3764ff9640 1 -- 172.21.15.44:0/190726718 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c040070 con 0x7f3768070ca0 2024-04-22T06:32:07.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.235+0000 7f3764ff9640 1 -- 172.21.15.44:0/190726718 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f375c0399a0 con 0x7f3768070ca0 2024-04-22T06:32:07.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.235+0000 7f376e648640 1 -- 172.21.15.44:0/190726718 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37681f1940 con 0x7f3768070ca0 2024-04-22T06:32:07.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.235+0000 7f3764ff9640 1 -- 172.21.15.44:0/190726718 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c035040 con 0x7f3768070ca0 2024-04-22T06:32:07.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.235+0000 7f376e648640 1 -- 172.21.15.44:0/190726718 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37681f1f00 con 0x7f3768070ca0 2024-04-22T06:32:07.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.236+0000 7f376e648640 1 -- 172.21.15.44:0/190726718 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37681f2100 con 0x7f3768070ca0 2024-04-22T06:32:07.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.237+0000 7f3764ff9640 1 -- 172.21.15.44:0/190726718 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f375c038020 con 0x7f3768070ca0 2024-04-22T06:32:07.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.237+0000 7f3764ff9640 1 --2- 172.21.15.44:0/190726718 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f375406c920 0x7f375406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:07.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.237+0000 7f3764ff9640 1 -- 172.21.15.44:0/190726718 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f375c0ad6d0 con 0x7f3768070ca0 2024-04-22T06:32:07.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.237+0000 7f3767fff640 1 --2- 172.21.15.44:0/190726718 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f375406c920 0x7f375406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:07.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.238+0000 7f3767fff640 1 --2- 172.21.15.44:0/190726718 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f375406c920 0x7f375406ede0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3758006fd0 tx=0x7f3758008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:07.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.241+0000 7f3764ff9640 1 -- 172.21.15.44:0/190726718 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f37681f2100 con 0x7f3768070ca0 2024-04-22T06:32:07.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.427+0000 7f376e648640 1 -- 172.21.15.44:0/190726718 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f3768069310 con 0x7f3768070ca0 2024-04-22T06:32:07.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.428+0000 7f3764ff9640 1 -- 172.21.15.44:0/190726718 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f375c07b0c0 con 0x7f3768070ca0 2024-04-22T06:32:07.427 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-04-22T06:32:07.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.429+0000 7f376e648640 1 -- 172.21.15.44:0/190726718 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f375406c920 msgr2=0x7f375406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:07.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.429+0000 7f376e648640 1 --2- 172.21.15.44:0/190726718 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f375406c920 0x7f375406ede0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3758006fd0 tx=0x7f3758008040 comp rx=0 tx=0).stop 2024-04-22T06:32:07.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.429+0000 7f376e648640 1 -- 172.21.15.44:0/190726718 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3768070ca0 msgr2=0x7f3768147500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:07.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.429+0000 7f376e648640 1 --2- 172.21.15.44:0/190726718 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3768070ca0 0x7f3768147500 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f375c034040 tx=0x7f375c0046c0 comp rx=0 tx=0).stop 2024-04-22T06:32:07.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.429+0000 7f376e648640 1 -- 172.21.15.44:0/190726718 shutdown_connections 2024-04-22T06:32:07.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.429+0000 7f376e648640 1 --2- 172.21.15.44:0/190726718 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f375406c920 0x7f375406ede0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:07.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.429+0000 7f376e648640 1 --2- 172.21.15.44:0/190726718 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37681551e0 0x7f3768158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:07.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.429+0000 7f376e648640 1 --2- 172.21.15.44:0/190726718 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3768070ca0 0x7f3768147500 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:07.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.429+0000 7f376e648640 1 --2- 172.21.15.44:0/190726718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f376806b930 0x7f37681f5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:07.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.429+0000 7f376e648640 1 -- 172.21.15.44:0/190726718 >> 172.21.15.44:0/190726718 conn(0x7f37680ce700 msgr2=0x7f3768153f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:07.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.430+0000 7f376e648640 1 -- 172.21.15.44:0/190726718 shutdown_connections 2024-04-22T06:32:07.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.430+0000 7f376e648640 1 -- 172.21.15.44:0/190726718 wait complete. 2024-04-22T06:32:07.439 INFO:tasks.quiescer.ceph_manager:waiting for clean 2024-04-22T06:32:07.439 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T06:32:07.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.541+0000 7f5b88e55640 1 Processor -- start 2024-04-22T06:32:07.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.541+0000 7f5b88e55640 1 -- start start 2024-04-22T06:32:07.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.542+0000 7f5b88e55640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f5b841552a0 0x7f5b8415bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:07.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.542+0000 7f5b88e55640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5b84155680 0x7f5b840cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:07.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.542+0000 7f5b88e55640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5b840cc310 0x7f5b840ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:07.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.542+0000 7f5b88e55640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f5b840cb990 con 0x7f5b841552a0 2024-04-22T06:32:07.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.542+0000 7f5b88e55640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f5b840cc760 con 0x7f5b840cc310 2024-04-22T06:32:07.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.542+0000 7f5b88e55640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f5b8406c6e0 con 0x7f5b84155680 2024-04-22T06:32:07.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.542+0000 7f5b8259c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f5b841552a0 0x7f5b8415bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:07.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.542+0000 7f5b8259c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f5b841552a0 0x7f5b8415bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52470/0 (socket says 172.21.15.44:52470) 2024-04-22T06:32:07.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.542+0000 7f5b8259c640 1 -- 172.21.15.44:0/1542120306 learned_addr learned my addr 172.21.15.44:0/1542120306 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:07.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.542+0000 7f5b81d9b640 1 --2- 172.21.15.44:0/1542120306 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5b84155680 0x7f5b840cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:07.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.542+0000 7f5b82d9d640 1 --2- 172.21.15.44:0/1542120306 >> v2:172.21.15.70:3300/0 conn(0x7f5b840cc310 0x7f5b840ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:07.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.542+0000 7f5b8259c640 1 -- 172.21.15.44:0/1542120306 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5b84155680 msgr2=0x7f5b840cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:07.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.542+0000 7f5b8259c640 1 --2- 172.21.15.44:0/1542120306 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5b84155680 0x7f5b840cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:07.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.542+0000 7f5b8259c640 1 -- 172.21.15.44:0/1542120306 >> v2:172.21.15.70:3300/0 conn(0x7f5b840cc310 msgr2=0x7f5b840ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:07.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.542+0000 7f5b8259c640 1 --2- 172.21.15.44:0/1542120306 >> v2:172.21.15.70:3300/0 conn(0x7f5b840cc310 0x7f5b840ccfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:07.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.542+0000 7f5b8259c640 1 -- 172.21.15.44:0/1542120306 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8406c9c0 con 0x7f5b841552a0 2024-04-22T06:32:07.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.542+0000 7f5b82d9d640 1 --2- 172.21.15.44:0/1542120306 >> v2:172.21.15.70:3300/0 conn(0x7f5b840cc310 0x7f5b840ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:32:07.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.543+0000 7f5b8259c640 1 --2- 172.21.15.44:0/1542120306 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b841552a0 0x7f5b8415bb80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5b84072f90 tx=0x7f5b7c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9489e1fd7e81ec7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:07.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.543+0000 7f5b8159a640 1 -- 172.21.15.44:0/1542120306 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c034070 con 0x7f5b841552a0 2024-04-22T06:32:07.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.543+0000 7f5b8159a640 1 -- 172.21.15.44:0/1542120306 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b7c030860 con 0x7f5b841552a0 2024-04-22T06:32:07.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.543+0000 7f5b8159a640 1 -- 172.21.15.44:0/1542120306 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c030c30 con 0x7f5b841552a0 2024-04-22T06:32:07.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.544+0000 7f5b88e55640 1 -- 172.21.15.44:0/1542120306 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b841552a0 msgr2=0x7f5b8415bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:07.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.544+0000 7f5b88e55640 1 --2- 172.21.15.44:0/1542120306 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b841552a0 0x7f5b8415bb80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5b84072f90 tx=0x7f5b7c02b9a0 comp rx=0 tx=0).stop 2024-04-22T06:32:07.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.544+0000 7f5b88e55640 1 -- 172.21.15.44:0/1542120306 shutdown_connections 2024-04-22T06:32:07.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.544+0000 7f5b88e55640 1 --2- 172.21.15.44:0/1542120306 >> v2:172.21.15.70:3300/0 conn(0x7f5b840cc310 0x7f5b840ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:07.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.544+0000 7f5b88e55640 1 --2- 172.21.15.44:0/1542120306 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5b84155680 0x7f5b840cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:07.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.544+0000 7f5b88e55640 1 --2- 172.21.15.44:0/1542120306 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b841552a0 0x7f5b8415bb80 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:07.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.544+0000 7f5b88e55640 1 -- 172.21.15.44:0/1542120306 >> 172.21.15.44:0/1542120306 conn(0x7f5b84075820 msgr2=0x7f5b84073000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:07.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.544+0000 7f5b88e55640 1 -- 172.21.15.44:0/1542120306 shutdown_connections 2024-04-22T06:32:07.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.544+0000 7f5b88e55640 1 -- 172.21.15.44:0/1542120306 wait complete. 2024-04-22T06:32:07.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.545+0000 7f5b88e55640 1 Processor -- start 2024-04-22T06:32:07.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.545+0000 7f5b88e55640 1 -- start start 2024-04-22T06:32:07.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.546+0000 7f5b88e55640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b840cc310 0x7f5b8415cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:07.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.546+0000 7f5b88e55640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b841552a0 0x7f5b841f3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:07.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.546+0000 7f5b88e55640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5b84150a50 0x7f5b84159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:07.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.546+0000 7f5b88e55640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8406d260 con 0x7f5b841552a0 2024-04-22T06:32:07.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.546+0000 7f5b88e55640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8406cf80 con 0x7f5b840cc310 2024-04-22T06:32:07.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.546+0000 7f5b88e55640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f5b8406d540 con 0x7f5b84150a50 2024-04-22T06:32:07.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.546+0000 7f5b81d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b841552a0 0x7f5b841f3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:07.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.546+0000 7f5b81d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b841552a0 0x7f5b841f3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52484/0 (socket says 172.21.15.44:52484) 2024-04-22T06:32:07.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.546+0000 7f5b81d9b640 1 -- 172.21.15.44:0/3407041517 learned_addr learned my addr 172.21.15.44:0/3407041517 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:07.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.546+0000 7f5b8259c640 1 --2- 172.21.15.44:0/3407041517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b840cc310 0x7f5b8415cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:07.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.546+0000 7f5b82d9d640 1 --2- 172.21.15.44:0/3407041517 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5b84150a50 0x7f5b84159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:07.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.546+0000 7f5b81d9b640 1 -- 172.21.15.44:0/3407041517 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5b84150a50 msgr2=0x7f5b84159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:07.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.546+0000 7f5b81d9b640 1 --2- 172.21.15.44:0/3407041517 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5b84150a50 0x7f5b84159cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:07.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.546+0000 7f5b81d9b640 1 -- 172.21.15.44:0/3407041517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b840cc310 msgr2=0x7f5b8415cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:07.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.546+0000 7f5b81d9b640 1 --2- 172.21.15.44:0/3407041517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b840cc310 0x7f5b8415cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:07.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.546+0000 7f5b81d9b640 1 -- 172.21.15.44:0/3407041517 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b840c6b60 con 0x7f5b841552a0 2024-04-22T06:32:07.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.546+0000 7f5b8259c640 1 --2- 172.21.15.44:0/3407041517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b840cc310 0x7f5b8415cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:32:07.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.547+0000 7f5b81d9b640 1 --2- 172.21.15.44:0/3407041517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b841552a0 0x7f5b841f3df0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5b7800d5f0 tx=0x7f5b7800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:07.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.547+0000 7f5b72ffd640 1 -- 172.21.15.44:0/3407041517 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b780125f0 con 0x7f5b841552a0 2024-04-22T06:32:07.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.547+0000 7f5b72ffd640 1 -- 172.21.15.44:0/3407041517 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b78019070 con 0x7f5b841552a0 2024-04-22T06:32:07.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.547+0000 7f5b88e55640 1 -- 172.21.15.44:0/3407041517 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b841f0880 con 0x7f5b841552a0 2024-04-22T06:32:07.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.547+0000 7f5b72ffd640 1 -- 172.21.15.44:0/3407041517 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b780129d0 con 0x7f5b841552a0 2024-04-22T06:32:07.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.547+0000 7f5b88e55640 1 -- 172.21.15.44:0/3407041517 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b840c7100 con 0x7f5b841552a0 2024-04-22T06:32:07.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.548+0000 7f5b88e55640 1 -- 172.21.15.44:0/3407041517 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b84068cc0 con 0x7f5b841552a0 2024-04-22T06:32:07.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.548+0000 7f5b72ffd640 1 -- 172.21.15.44:0/3407041517 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5b78012b70 con 0x7f5b841552a0 2024-04-22T06:32:07.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.549+0000 7f5b72ffd640 1 --2- 172.21.15.44:0/3407041517 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f5b6406c920 0x7f5b6406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:07.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.549+0000 7f5b72ffd640 1 -- 172.21.15.44:0/3407041517 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f5b7808c390 con 0x7f5b841552a0 2024-04-22T06:32:07.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.549+0000 7f5b8259c640 1 --2- 172.21.15.44:0/3407041517 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f5b6406c920 0x7f5b6406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:07.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.549+0000 7f5b8259c640 1 --2- 172.21.15.44:0/3407041517 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f5b6406c920 0x7f5b6406ede0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c036000 tx=0x7f5b7c03b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:07.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.553+0000 7f5b72ffd640 1 -- 172.21.15.44:0/3407041517 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5b7805a320 con 0x7f5b841552a0 2024-04-22T06:32:07.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.736+0000 7f5b88e55640 1 -- 172.21.15.44:0/3407041517 --> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5b840ca090 con 0x7f5b6406c920 2024-04-22T06:32:07.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.745+0000 7f5b72ffd640 1 -- 172.21.15.44:0/3407041517 <== mgr.4111 v2:172.21.15.70:6832/2570466131 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332632 (secure 0 0 0) 0x7f5b840ca090 con 0x7f5b6406c920 2024-04-22T06:32:07.747 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:32:07.749 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-04-22T06:32:07.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.748+0000 7f5b88e55640 1 -- 172.21.15.44:0/3407041517 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f5b6406c920 msgr2=0x7f5b6406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:07.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.748+0000 7f5b88e55640 1 --2- 172.21.15.44:0/3407041517 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f5b6406c920 0x7f5b6406ede0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c036000 tx=0x7f5b7c03b040 comp rx=0 tx=0).stop 2024-04-22T06:32:07.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.748+0000 7f5b88e55640 1 -- 172.21.15.44:0/3407041517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b841552a0 msgr2=0x7f5b841f3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:07.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.748+0000 7f5b88e55640 1 --2- 172.21.15.44:0/3407041517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b841552a0 0x7f5b841f3df0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5b7800d5f0 tx=0x7f5b7800dab0 comp rx=0 tx=0).stop 2024-04-22T06:32:07.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.748+0000 7f5b88e55640 1 -- 172.21.15.44:0/3407041517 shutdown_connections 2024-04-22T06:32:07.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.748+0000 7f5b88e55640 1 --2- 172.21.15.44:0/3407041517 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f5b6406c920 0x7f5b6406ede0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:07.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.748+0000 7f5b88e55640 1 --2- 172.21.15.44:0/3407041517 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5b84150a50 0x7f5b84159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:07.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.748+0000 7f5b88e55640 1 --2- 172.21.15.44:0/3407041517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b841552a0 0x7f5b841f3df0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:07.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.748+0000 7f5b88e55640 1 --2- 172.21.15.44:0/3407041517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b840cc310 0x7f5b8415cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:07.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.748+0000 7f5b88e55640 1 -- 172.21.15.44:0/3407041517 >> 172.21.15.44:0/3407041517 conn(0x7f5b84075820 msgr2=0x7f5b840c88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:07.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.748+0000 7f5b88e55640 1 -- 172.21.15.44:0/3407041517 shutdown_connections 2024-04-22T06:32:07.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.748+0000 7f5b88e55640 1 -- 172.21.15.44:0/3407041517 wait complete. 2024-04-22T06:32:07.758 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":57,"stamp":"2024-04-22T06:32:06.558361+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600632,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":367,"num_write_kb":2482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":299,"ondisk_log_size":299,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236456,"kb_used_data":3688,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738264,"statfs":{"total":773094113280,"available":772851982336,"internally_reserved":0,"allocated":3776512,"data_stored":2153584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1406,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002522"},"pg_stats":[{"pgid":"2.1f","version":"27'13","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:55.161180+0000","last_change":"2024-04-22T06:31:06.002169+0000","last_active":"2024-04-22T06:31:55.161180+0000","last_peered":"2024-04-22T06:31:55.161180+0000","last_clean":"2024-04-22T06:31:55.161180+0000","last_became_active":"2024-04-22T06:31:06.001899+0000","last_became_peered":"2024-04-22T06:31:06.001899+0000","last_unstale":"2024-04-22T06:31:55.161180+0000","last_undegraded":"2024-04-22T06:31:55.161180+0000","last_fullsized":"2024-04-22T06:31:55.161180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:11:48.056404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.146682+0000","last_change":"2024-04-22T06:31:07.025982+0000","last_active":"2024-04-22T06:31:48.146682+0000","last_peered":"2024-04-22T06:31:48.146682+0000","last_clean":"2024-04-22T06:31:48.146682+0000","last_became_active":"2024-04-22T06:31:07.025650+0000","last_became_peered":"2024-04-22T06:31:07.025650+0000","last_unstale":"2024-04-22T06:31:48.146682+0000","last_undegraded":"2024-04-22T06:31:48.146682+0000","last_fullsized":"2024-04-22T06:31:48.146682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:58:58.029898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145570+0000","last_change":"2024-04-22T06:31:11.048083+0000","last_active":"2024-04-22T06:31:48.145570+0000","last_peered":"2024-04-22T06:31:48.145570+0000","last_clean":"2024-04-22T06:31:48.145570+0000","last_became_active":"2024-04-22T06:31:11.047682+0000","last_became_peered":"2024-04-22T06:31:11.047682+0000","last_unstale":"2024-04-22T06:31:48.145570+0000","last_undegraded":"2024-04-22T06:31:48.145570+0000","last_fullsized":"2024-04-22T06:31:48.145570+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:34:30.158935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"25'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.146793+0000","last_change":"2024-04-22T06:31:06.007484+0000","last_active":"2024-04-22T06:31:48.146793+0000","last_peered":"2024-04-22T06:31:48.146793+0000","last_clean":"2024-04-22T06:31:48.146793+0000","last_became_active":"2024-04-22T06:31:06.007204+0000","last_became_peered":"2024-04-22T06:31:06.007204+0000","last_unstale":"2024-04-22T06:31:48.146793+0000","last_undegraded":"2024-04-22T06:31:48.146793+0000","last_fullsized":"2024-04-22T06:31:48.146793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:39:18.905813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142625+0000","last_change":"2024-04-22T06:31:07.014658+0000","last_active":"2024-04-22T06:31:48.142625+0000","last_peered":"2024-04-22T06:31:48.142625+0000","last_clean":"2024-04-22T06:31:48.142625+0000","last_became_active":"2024-04-22T06:31:07.014283+0000","last_became_peered":"2024-04-22T06:31:07.014283+0000","last_unstale":"2024-04-22T06:31:48.142625+0000","last_undegraded":"2024-04-22T06:31:48.142625+0000","last_fullsized":"2024-04-22T06:31:48.142625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:43:01.159439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.190754+0000","last_change":"2024-04-22T06:31:11.045114+0000","last_active":"2024-04-22T06:31:48.190754+0000","last_peered":"2024-04-22T06:31:48.190754+0000","last_clean":"2024-04-22T06:31:48.190754+0000","last_became_active":"2024-04-22T06:31:11.044728+0000","last_became_peered":"2024-04-22T06:31:11.044728+0000","last_unstale":"2024-04-22T06:31:48.190754+0000","last_undegraded":"2024-04-22T06:31:48.190754+0000","last_fullsized":"2024-04-22T06:31:48.190754+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:48:19.215184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"27'12","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:55.161553+0000","last_change":"2024-04-22T06:31:05.995468+0000","last_active":"2024-04-22T06:31:55.161553+0000","last_peered":"2024-04-22T06:31:55.161553+0000","last_clean":"2024-04-22T06:31:55.161553+0000","last_became_active":"2024-04-22T06:31:05.995159+0000","last_became_peered":"2024-04-22T06:31:05.995159+0000","last_unstale":"2024-04-22T06:31:55.161553+0000","last_undegraded":"2024-04-22T06:31:55.161553+0000","last_fullsized":"2024-04-22T06:31:55.161553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:01:12.223718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143359+0000","last_change":"2024-04-22T06:31:07.020024+0000","last_active":"2024-04-22T06:31:48.143359+0000","last_peered":"2024-04-22T06:31:48.143359+0000","last_clean":"2024-04-22T06:31:48.143359+0000","last_became_active":"2024-04-22T06:31:07.019490+0000","last_became_peered":"2024-04-22T06:31:07.019490+0000","last_unstale":"2024-04-22T06:31:48.143359+0000","last_undegraded":"2024-04-22T06:31:48.143359+0000","last_fullsized":"2024-04-22T06:31:48.143359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:26:41.811978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.191549+0000","last_change":"2024-04-22T06:31:11.057556+0000","last_active":"2024-04-22T06:31:48.191549+0000","last_peered":"2024-04-22T06:31:48.191549+0000","last_clean":"2024-04-22T06:31:48.191549+0000","last_became_active":"2024-04-22T06:31:11.057174+0000","last_became_peered":"2024-04-22T06:31:11.057174+0000","last_unstale":"2024-04-22T06:31:48.191549+0000","last_undegraded":"2024-04-22T06:31:48.191549+0000","last_fullsized":"2024-04-22T06:31:48.191549+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:41:19.526548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"27'5","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:55.160730+0000","last_change":"2024-04-22T06:31:05.996281+0000","last_active":"2024-04-22T06:31:55.160730+0000","last_peered":"2024-04-22T06:31:55.160730+0000","last_clean":"2024-04-22T06:31:55.160730+0000","last_became_active":"2024-04-22T06:31:05.995951+0000","last_became_peered":"2024-04-22T06:31:05.995951+0000","last_unstale":"2024-04-22T06:31:55.160730+0000","last_undegraded":"2024-04-22T06:31:55.160730+0000","last_fullsized":"2024-04-22T06:31:55.160730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:01:43.732947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143274+0000","last_change":"2024-04-22T06:31:07.017755+0000","last_active":"2024-04-22T06:31:48.143274+0000","last_peered":"2024-04-22T06:31:48.143274+0000","last_clean":"2024-04-22T06:31:48.143274+0000","last_became_active":"2024-04-22T06:31:07.017420+0000","last_became_peered":"2024-04-22T06:31:07.017420+0000","last_unstale":"2024-04-22T06:31:48.143274+0000","last_undegraded":"2024-04-22T06:31:48.143274+0000","last_fullsized":"2024-04-22T06:31:48.143274+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:30:57.913020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.191016+0000","last_change":"2024-04-22T06:31:11.049497+0000","last_active":"2024-04-22T06:31:48.191016+0000","last_peered":"2024-04-22T06:31:48.191016+0000","last_clean":"2024-04-22T06:31:48.191016+0000","last_became_active":"2024-04-22T06:31:11.049079+0000","last_became_peered":"2024-04-22T06:31:11.049079+0000","last_unstale":"2024-04-22T06:31:48.191016+0000","last_undegraded":"2024-04-22T06:31:48.191016+0000","last_fullsized":"2024-04-22T06:31:48.191016+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:29:02.299691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145065+0000","last_change":"2024-04-22T06:31:07.023043+0000","last_active":"2024-04-22T06:31:48.145065+0000","last_peered":"2024-04-22T06:31:48.145065+0000","last_clean":"2024-04-22T06:31:48.145065+0000","last_became_active":"2024-04-22T06:31:07.022747+0000","last_became_peered":"2024-04-22T06:31:07.022747+0000","last_unstale":"2024-04-22T06:31:48.145065+0000","last_undegraded":"2024-04-22T06:31:48.145065+0000","last_fullsized":"2024-04-22T06:31:48.145065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:52:33.626072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"25'4","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143096+0000","last_change":"2024-04-22T06:31:05.997936+0000","last_active":"2024-04-22T06:31:48.143096+0000","last_peered":"2024-04-22T06:31:48.143096+0000","last_clean":"2024-04-22T06:31:48.143096+0000","last_became_active":"2024-04-22T06:31:05.997618+0000","last_became_peered":"2024-04-22T06:31:05.997618+0000","last_unstale":"2024-04-22T06:31:48.143096+0000","last_undegraded":"2024-04-22T06:31:48.143096+0000","last_fullsized":"2024-04-22T06:31:48.143096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:43:00.396691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.191374+0000","last_change":"2024-04-22T06:31:11.047725+0000","last_active":"2024-04-22T06:31:48.191374+0000","last_peered":"2024-04-22T06:31:48.191374+0000","last_clean":"2024-04-22T06:31:48.191374+0000","last_became_active":"2024-04-22T06:31:11.047278+0000","last_became_peered":"2024-04-22T06:31:11.047278+0000","last_unstale":"2024-04-22T06:31:48.191374+0000","last_undegraded":"2024-04-22T06:31:48.191374+0000","last_fullsized":"2024-04-22T06:31:48.191374+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:32:56.201059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145944+0000","last_change":"2024-04-22T06:31:07.021250+0000","last_active":"2024-04-22T06:31:48.145944+0000","last_peered":"2024-04-22T06:31:48.145944+0000","last_clean":"2024-04-22T06:31:48.145944+0000","last_became_active":"2024-04-22T06:31:07.020876+0000","last_became_peered":"2024-04-22T06:31:07.020876+0000","last_unstale":"2024-04-22T06:31:48.145944+0000","last_undegraded":"2024-04-22T06:31:48.145944+0000","last_fullsized":"2024-04-22T06:31:48.145944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:21:31.210819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"27'3","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:49.142316+0000","last_change":"2024-04-22T06:31:06.012775+0000","last_active":"2024-04-22T06:31:49.142316+0000","last_peered":"2024-04-22T06:31:49.142316+0000","last_clean":"2024-04-22T06:31:49.142316+0000","last_became_active":"2024-04-22T06:31:06.012474+0000","last_became_peered":"2024-04-22T06:31:06.012474+0000","last_unstale":"2024-04-22T06:31:49.142316+0000","last_undegraded":"2024-04-22T06:31:49.142316+0000","last_fullsized":"2024-04-22T06:31:49.142316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:09:48.197011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.191217+0000","last_change":"2024-04-22T06:31:11.057641+0000","last_active":"2024-04-22T06:31:48.191217+0000","last_peered":"2024-04-22T06:31:48.191217+0000","last_clean":"2024-04-22T06:31:48.191217+0000","last_became_active":"2024-04-22T06:31:11.057221+0000","last_became_peered":"2024-04-22T06:31:11.057221+0000","last_unstale":"2024-04-22T06:31:48.191217+0000","last_undegraded":"2024-04-22T06:31:48.191217+0000","last_fullsized":"2024-04-22T06:31:48.191217+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:14:57.713113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145398+0000","last_change":"2024-04-22T06:31:07.020822+0000","last_active":"2024-04-22T06:31:48.145398+0000","last_peered":"2024-04-22T06:31:48.145398+0000","last_clean":"2024-04-22T06:31:48.145398+0000","last_became_active":"2024-04-22T06:31:07.020484+0000","last_became_peered":"2024-04-22T06:31:07.020484+0000","last_unstale":"2024-04-22T06:31:48.145398+0000","last_undegraded":"2024-04-22T06:31:48.145398+0000","last_fullsized":"2024-04-22T06:31:48.145398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:18:26.522131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"27'7","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:55.160232+0000","last_change":"2024-04-22T06:31:06.000000+0000","last_active":"2024-04-22T06:31:55.160232+0000","last_peered":"2024-04-22T06:31:55.160232+0000","last_clean":"2024-04-22T06:31:55.160232+0000","last_became_active":"2024-04-22T06:31:05.999663+0000","last_became_peered":"2024-04-22T06:31:05.999663+0000","last_unstale":"2024-04-22T06:31:55.160232+0000","last_undegraded":"2024-04-22T06:31:55.160232+0000","last_fullsized":"2024-04-22T06:31:55.160232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:42.805084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.147239+0000","last_change":"2024-04-22T06:31:11.057991+0000","last_active":"2024-04-22T06:31:48.147239+0000","last_peered":"2024-04-22T06:31:48.147239+0000","last_clean":"2024-04-22T06:31:48.147239+0000","last_became_active":"2024-04-22T06:31:11.057595+0000","last_became_peered":"2024-04-22T06:31:11.057595+0000","last_unstale":"2024-04-22T06:31:48.147239+0000","last_undegraded":"2024-04-22T06:31:48.147239+0000","last_fullsized":"2024-04-22T06:31:48.147239+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:17:06.366663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.191026+0000","last_change":"2024-04-22T06:31:07.017303+0000","last_active":"2024-04-22T06:31:48.191026+0000","last_peered":"2024-04-22T06:31:48.191026+0000","last_clean":"2024-04-22T06:31:48.191026+0000","last_became_active":"2024-04-22T06:31:07.016990+0000","last_became_peered":"2024-04-22T06:31:07.016990+0000","last_unstale":"2024-04-22T06:31:48.191026+0000","last_undegraded":"2024-04-22T06:31:48.191026+0000","last_fullsized":"2024-04-22T06:31:48.191026+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:51:38.275472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"25'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143212+0000","last_change":"2024-04-22T06:31:05.997363+0000","last_active":"2024-04-22T06:31:48.143212+0000","last_peered":"2024-04-22T06:31:48.143212+0000","last_clean":"2024-04-22T06:31:48.143212+0000","last_became_active":"2024-04-22T06:31:05.996982+0000","last_became_peered":"2024-04-22T06:31:05.996982+0000","last_unstale":"2024-04-22T06:31:48.143212+0000","last_undegraded":"2024-04-22T06:31:48.143212+0000","last_fullsized":"2024-04-22T06:31:48.143212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:23:50.052130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.191397+0000","last_change":"2024-04-22T06:31:11.048507+0000","last_active":"2024-04-22T06:31:48.191397+0000","last_peered":"2024-04-22T06:31:48.191397+0000","last_clean":"2024-04-22T06:31:48.191397+0000","last_became_active":"2024-04-22T06:31:11.047722+0000","last_became_peered":"2024-04-22T06:31:11.047722+0000","last_unstale":"2024-04-22T06:31:48.191397+0000","last_undegraded":"2024-04-22T06:31:48.191397+0000","last_fullsized":"2024-04-22T06:31:48.191397+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:04:48.732029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143768+0000","last_change":"2024-04-22T06:31:07.021095+0000","last_active":"2024-04-22T06:31:48.143768+0000","last_peered":"2024-04-22T06:31:48.143768+0000","last_clean":"2024-04-22T06:31:48.143768+0000","last_became_active":"2024-04-22T06:31:07.020814+0000","last_became_peered":"2024-04-22T06:31:07.020814+0000","last_unstale":"2024-04-22T06:31:48.143768+0000","last_undegraded":"2024-04-22T06:31:48.143768+0000","last_fullsized":"2024-04-22T06:31:48.143768+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:27:08.869392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"27'7","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:55.160312+0000","last_change":"2024-04-22T06:31:05.997508+0000","last_active":"2024-04-22T06:31:55.160312+0000","last_peered":"2024-04-22T06:31:55.160312+0000","last_clean":"2024-04-22T06:31:55.160312+0000","last_became_active":"2024-04-22T06:31:05.997233+0000","last_became_peered":"2024-04-22T06:31:05.997233+0000","last_unstale":"2024-04-22T06:31:55.160312+0000","last_undegraded":"2024-04-22T06:31:55.160312+0000","last_fullsized":"2024-04-22T06:31:55.160312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:40:56.909549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.190833+0000","last_change":"2024-04-22T06:31:11.045424+0000","last_active":"2024-04-22T06:31:48.190833+0000","last_peered":"2024-04-22T06:31:48.190833+0000","last_clean":"2024-04-22T06:31:48.190833+0000","last_became_active":"2024-04-22T06:31:11.045061+0000","last_became_peered":"2024-04-22T06:31:11.045061+0000","last_unstale":"2024-04-22T06:31:48.190833+0000","last_undegraded":"2024-04-22T06:31:48.190833+0000","last_fullsized":"2024-04-22T06:31:48.190833+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:00:33.958921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.149855+0000","last_change":"2024-04-22T06:31:48.148715+0000","last_active":"2024-04-22T06:31:48.149855+0000","last_peered":"2024-04-22T06:31:48.149855+0000","last_clean":"2024-04-22T06:31:48.149855+0000","last_became_active":"2024-04-22T06:31:48.148435+0000","last_became_peered":"2024-04-22T06:31:48.148435+0000","last_unstale":"2024-04-22T06:31:48.149855+0000","last_undegraded":"2024-04-22T06:31:48.149855+0000","last_fullsized":"2024-04-22T06:31:48.149855+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:08:09.509939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"27'7","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:49.142408+0000","last_change":"2024-04-22T06:31:05.997990+0000","last_active":"2024-04-22T06:31:49.142408+0000","last_peered":"2024-04-22T06:31:49.142408+0000","last_clean":"2024-04-22T06:31:49.142408+0000","last_became_active":"2024-04-22T06:31:05.997352+0000","last_became_peered":"2024-04-22T06:31:05.997352+0000","last_unstale":"2024-04-22T06:31:49.142408+0000","last_undegraded":"2024-04-22T06:31:49.142408+0000","last_fullsized":"2024-04-22T06:31:49.142408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:15:40.726807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.144948+0000","last_change":"2024-04-22T06:31:11.053487+0000","last_active":"2024-04-22T06:31:48.144948+0000","last_peered":"2024-04-22T06:31:48.144948+0000","last_clean":"2024-04-22T06:31:48.144948+0000","last_became_active":"2024-04-22T06:31:11.053078+0000","last_became_peered":"2024-04-22T06:31:11.053078+0000","last_unstale":"2024-04-22T06:31:48.144948+0000","last_undegraded":"2024-04-22T06:31:48.144948+0000","last_fullsized":"2024-04-22T06:31:48.144948+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:23:16.130369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.147628+0000","last_change":"2024-04-22T06:31:07.020420+0000","last_active":"2024-04-22T06:31:48.147628+0000","last_peered":"2024-04-22T06:31:48.147628+0000","last_clean":"2024-04-22T06:31:48.147628+0000","last_became_active":"2024-04-22T06:31:07.020110+0000","last_became_peered":"2024-04-22T06:31:07.020110+0000","last_unstale":"2024-04-22T06:31:48.147628+0000","last_undegraded":"2024-04-22T06:31:48.147628+0000","last_fullsized":"2024-04-22T06:31:48.147628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:44:23.997057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"27'4","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:49.144779+0000","last_change":"2024-04-22T06:31:05.997357+0000","last_active":"2024-04-22T06:31:49.144779+0000","last_peered":"2024-04-22T06:31:49.144779+0000","last_clean":"2024-04-22T06:31:49.144779+0000","last_became_active":"2024-04-22T06:31:05.996913+0000","last_became_peered":"2024-04-22T06:31:05.996913+0000","last_unstale":"2024-04-22T06:31:49.144779+0000","last_undegraded":"2024-04-22T06:31:49.144779+0000","last_fullsized":"2024-04-22T06:31:49.144779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:50:51.880199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142598+0000","last_change":"2024-04-22T06:31:11.059735+0000","last_active":"2024-04-22T06:31:48.142598+0000","last_peered":"2024-04-22T06:31:48.142598+0000","last_clean":"2024-04-22T06:31:48.142598+0000","last_became_active":"2024-04-22T06:31:11.059325+0000","last_became_peered":"2024-04-22T06:31:11.059325+0000","last_unstale":"2024-04-22T06:31:48.142598+0000","last_undegraded":"2024-04-22T06:31:48.142598+0000","last_fullsized":"2024-04-22T06:31:48.142598+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:59:52.881783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143372+0000","last_change":"2024-04-22T06:31:07.026576+0000","last_active":"2024-04-22T06:31:48.143372+0000","last_peered":"2024-04-22T06:31:48.143372+0000","last_clean":"2024-04-22T06:31:48.143372+0000","last_became_active":"2024-04-22T06:31:07.026057+0000","last_became_peered":"2024-04-22T06:31:07.026057+0000","last_unstale":"2024-04-22T06:31:48.143372+0000","last_undegraded":"2024-04-22T06:31:48.143372+0000","last_fullsized":"2024-04-22T06:31:48.143372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:49:32.525918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"27'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.147781+0000","last_change":"2024-04-22T06:31:06.009043+0000","last_active":"2024-04-22T06:31:48.147781+0000","last_peered":"2024-04-22T06:31:48.147781+0000","last_clean":"2024-04-22T06:31:48.147781+0000","last_became_active":"2024-04-22T06:31:06.008752+0000","last_became_peered":"2024-04-22T06:31:06.008752+0000","last_unstale":"2024-04-22T06:31:48.147781+0000","last_undegraded":"2024-04-22T06:31:48.147781+0000","last_fullsized":"2024-04-22T06:31:48.147781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:52:34.706288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.193611+0000","last_change":"2024-04-22T06:31:48.192229+0000","last_active":"2024-04-22T06:31:48.193611+0000","last_peered":"2024-04-22T06:31:48.193611+0000","last_clean":"2024-04-22T06:31:48.193611+0000","last_became_active":"2024-04-22T06:31:48.191700+0000","last_became_peered":"2024-04-22T06:31:48.191700+0000","last_unstale":"2024-04-22T06:31:48.193611+0000","last_undegraded":"2024-04-22T06:31:48.193611+0000","last_fullsized":"2024-04-22T06:31:48.193611+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:45:48.217879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4,0],"acting":[2,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.191316+0000","last_change":"2024-04-22T06:31:07.028506+0000","last_active":"2024-04-22T06:31:48.191316+0000","last_peered":"2024-04-22T06:31:48.191316+0000","last_clean":"2024-04-22T06:31:48.191316+0000","last_became_active":"2024-04-22T06:31:07.028183+0000","last_became_peered":"2024-04-22T06:31:07.028183+0000","last_unstale":"2024-04-22T06:31:48.191316+0000","last_undegraded":"2024-04-22T06:31:48.191316+0000","last_fullsized":"2024-04-22T06:31:48.191316+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:38:45.965934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"27'5","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:55.162069+0000","last_change":"2024-04-22T06:31:06.002052+0000","last_active":"2024-04-22T06:31:55.162069+0000","last_peered":"2024-04-22T06:31:55.162069+0000","last_clean":"2024-04-22T06:31:55.162069+0000","last_became_active":"2024-04-22T06:31:06.000645+0000","last_became_peered":"2024-04-22T06:31:06.000645+0000","last_unstale":"2024-04-22T06:31:55.162069+0000","last_undegraded":"2024-04-22T06:31:55.162069+0000","last_fullsized":"2024-04-22T06:31:55.162069+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:55:38.982563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.147626+0000","last_change":"2024-04-22T06:31:11.048333+0000","last_active":"2024-04-22T06:31:48.147626+0000","last_peered":"2024-04-22T06:31:48.147626+0000","last_clean":"2024-04-22T06:31:48.147626+0000","last_became_active":"2024-04-22T06:31:11.047913+0000","last_became_peered":"2024-04-22T06:31:11.047913+0000","last_unstale":"2024-04-22T06:31:48.147626+0000","last_undegraded":"2024-04-22T06:31:48.147626+0000","last_fullsized":"2024-04-22T06:31:48.147626+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:11:52.622436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.147090+0000","last_change":"2024-04-22T06:31:07.017430+0000","last_active":"2024-04-22T06:31:48.147090+0000","last_peered":"2024-04-22T06:31:48.147090+0000","last_clean":"2024-04-22T06:31:48.147090+0000","last_became_active":"2024-04-22T06:31:07.017112+0000","last_became_peered":"2024-04-22T06:31:07.017112+0000","last_unstale":"2024-04-22T06:31:48.147090+0000","last_undegraded":"2024-04-22T06:31:48.147090+0000","last_fullsized":"2024-04-22T06:31:48.147090+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:52:18.098301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"27'5","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145524+0000","last_change":"2024-04-22T06:31:06.002202+0000","last_active":"2024-04-22T06:31:48.145524+0000","last_peered":"2024-04-22T06:31:48.145524+0000","last_clean":"2024-04-22T06:31:48.145524+0000","last_became_active":"2024-04-22T06:31:06.001916+0000","last_became_peered":"2024-04-22T06:31:06.001916+0000","last_unstale":"2024-04-22T06:31:48.145524+0000","last_undegraded":"2024-04-22T06:31:48.145524+0000","last_fullsized":"2024-04-22T06:31:48.145524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:35:52.485268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.147336+0000","last_change":"2024-04-22T06:31:11.054576+0000","last_active":"2024-04-22T06:31:48.147336+0000","last_peered":"2024-04-22T06:31:48.147336+0000","last_clean":"2024-04-22T06:31:48.147336+0000","last_became_active":"2024-04-22T06:31:11.054158+0000","last_became_peered":"2024-04-22T06:31:11.054158+0000","last_unstale":"2024-04-22T06:31:48.147336+0000","last_undegraded":"2024-04-22T06:31:48.147336+0000","last_fullsized":"2024-04-22T06:31:48.147336+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:48:50.809440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142347+0000","last_change":"2024-04-22T06:31:07.023064+0000","last_active":"2024-04-22T06:31:48.142347+0000","last_peered":"2024-04-22T06:31:48.142347+0000","last_clean":"2024-04-22T06:31:48.142347+0000","last_became_active":"2024-04-22T06:31:07.022753+0000","last_became_peered":"2024-04-22T06:31:07.022753+0000","last_unstale":"2024-04-22T06:31:48.142347+0000","last_undegraded":"2024-04-22T06:31:48.142347+0000","last_fullsized":"2024-04-22T06:31:48.142347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:25:38.039689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"27'4","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143372+0000","last_change":"2024-04-22T06:31:05.998384+0000","last_active":"2024-04-22T06:31:48.143372+0000","last_peered":"2024-04-22T06:31:48.143372+0000","last_clean":"2024-04-22T06:31:48.143372+0000","last_became_active":"2024-04-22T06:31:05.997761+0000","last_became_peered":"2024-04-22T06:31:05.997761+0000","last_unstale":"2024-04-22T06:31:48.143372+0000","last_undegraded":"2024-04-22T06:31:48.143372+0000","last_fullsized":"2024-04-22T06:31:48.143372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:07:12.014505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.149828+0000","last_change":"2024-04-22T06:31:48.149828+0000","last_active":"2024-04-22T06:31:48.149828+0000","last_peered":"2024-04-22T06:31:48.149828+0000","last_clean":"2024-04-22T06:31:48.149828+0000","last_became_active":"2024-04-22T06:31:48.149336+0000","last_became_peered":"2024-04-22T06:31:48.149336+0000","last_unstale":"2024-04-22T06:31:48.149828+0000","last_undegraded":"2024-04-22T06:31:48.149828+0000","last_fullsized":"2024-04-22T06:31:48.149828+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:09:53.968698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2,7],"acting":[0,5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145924+0000","last_change":"2024-04-22T06:31:07.026139+0000","last_active":"2024-04-22T06:31:48.145924+0000","last_peered":"2024-04-22T06:31:48.145924+0000","last_clean":"2024-04-22T06:31:48.145924+0000","last_became_active":"2024-04-22T06:31:07.025826+0000","last_became_peered":"2024-04-22T06:31:07.025826+0000","last_unstale":"2024-04-22T06:31:48.145924+0000","last_undegraded":"2024-04-22T06:31:48.145924+0000","last_fullsized":"2024-04-22T06:31:48.145924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:58:48.200581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"27'6","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:49.144921+0000","last_change":"2024-04-22T06:31:06.001815+0000","last_active":"2024-04-22T06:31:49.144921+0000","last_peered":"2024-04-22T06:31:49.144921+0000","last_clean":"2024-04-22T06:31:49.144921+0000","last_became_active":"2024-04-22T06:31:06.000629+0000","last_became_peered":"2024-04-22T06:31:06.000629+0000","last_unstale":"2024-04-22T06:31:49.144921+0000","last_undegraded":"2024-04-22T06:31:49.144921+0000","last_fullsized":"2024-04-22T06:31:49.144921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:49:31.100969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":182,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143419+0000","last_change":"2024-04-22T06:30:56.214982+0000","last_active":"2024-04-22T06:31:48.143419+0000","last_peered":"2024-04-22T06:31:48.143419+0000","last_clean":"2024-04-22T06:31:48.143419+0000","last_became_active":"2024-04-22T06:30:56.214024+0000","last_became_peered":"2024-04-22T06:30:56.214024+0000","last_unstale":"2024-04-22T06:31:48.143419+0000","last_undegraded":"2024-04-22T06:31:48.143419+0000","last_fullsized":"2024-04-22T06:31:48.143419+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:30:54.947156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:30:54.947156+0000","last_clean_scrub_stamp":"2024-04-22T06:30:54.947156+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:32:37.920238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145498+0000","last_change":"2024-04-22T06:31:11.040152+0000","last_active":"2024-04-22T06:31:48.145498+0000","last_peered":"2024-04-22T06:31:48.145498+0000","last_clean":"2024-04-22T06:31:48.145498+0000","last_became_active":"2024-04-22T06:31:11.039749+0000","last_became_peered":"2024-04-22T06:31:11.039749+0000","last_unstale":"2024-04-22T06:31:48.145498+0000","last_undegraded":"2024-04-22T06:31:48.145498+0000","last_fullsized":"2024-04-22T06:31:48.145498+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:31:23.103876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142265+0000","last_change":"2024-04-22T06:31:07.011062+0000","last_active":"2024-04-22T06:31:48.142265+0000","last_peered":"2024-04-22T06:31:48.142265+0000","last_clean":"2024-04-22T06:31:48.142265+0000","last_became_active":"2024-04-22T06:31:07.010755+0000","last_became_peered":"2024-04-22T06:31:07.010755+0000","last_unstale":"2024-04-22T06:31:48.142265+0000","last_undegraded":"2024-04-22T06:31:48.142265+0000","last_fullsized":"2024-04-22T06:31:48.142265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:57:01.373036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"27'5","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:49.141800+0000","last_change":"2024-04-22T06:31:05.998083+0000","last_active":"2024-04-22T06:31:49.141800+0000","last_peered":"2024-04-22T06:31:49.141800+0000","last_clean":"2024-04-22T06:31:49.141800+0000","last_became_active":"2024-04-22T06:31:05.997777+0000","last_became_peered":"2024-04-22T06:31:05.997777+0000","last_unstale":"2024-04-22T06:31:49.141800+0000","last_undegraded":"2024-04-22T06:31:49.141800+0000","last_fullsized":"2024-04-22T06:31:49.141800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:15:13.171198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145255+0000","last_change":"2024-04-22T06:31:11.049607+0000","last_active":"2024-04-22T06:31:48.145255+0000","last_peered":"2024-04-22T06:31:48.145255+0000","last_clean":"2024-04-22T06:31:48.145255+0000","last_became_active":"2024-04-22T06:31:11.049156+0000","last_became_peered":"2024-04-22T06:31:11.049156+0000","last_unstale":"2024-04-22T06:31:48.145255+0000","last_undegraded":"2024-04-22T06:31:48.145255+0000","last_fullsized":"2024-04-22T06:31:48.145255+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:15:48.599278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143439+0000","last_change":"2024-04-22T06:31:07.026306+0000","last_active":"2024-04-22T06:31:48.143439+0000","last_peered":"2024-04-22T06:31:48.143439+0000","last_clean":"2024-04-22T06:31:48.143439+0000","last_became_active":"2024-04-22T06:31:07.025987+0000","last_became_peered":"2024-04-22T06:31:07.025987+0000","last_unstale":"2024-04-22T06:31:48.143439+0000","last_undegraded":"2024-04-22T06:31:48.143439+0000","last_fullsized":"2024-04-22T06:31:48.143439+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:50:31.803180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"27'3","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:49.142263+0000","last_change":"2024-04-22T06:31:06.003115+0000","last_active":"2024-04-22T06:31:49.142263+0000","last_peered":"2024-04-22T06:31:49.142263+0000","last_clean":"2024-04-22T06:31:49.142263+0000","last_became_active":"2024-04-22T06:31:06.001945+0000","last_became_peered":"2024-04-22T06:31:06.001945+0000","last_unstale":"2024-04-22T06:31:49.142263+0000","last_undegraded":"2024-04-22T06:31:49.142263+0000","last_fullsized":"2024-04-22T06:31:49.142263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:59:25.770504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142789+0000","last_change":"2024-04-22T06:31:11.058841+0000","last_active":"2024-04-22T06:31:48.142789+0000","last_peered":"2024-04-22T06:31:48.142789+0000","last_clean":"2024-04-22T06:31:48.142789+0000","last_became_active":"2024-04-22T06:31:11.058407+0000","last_became_peered":"2024-04-22T06:31:11.058407+0000","last_unstale":"2024-04-22T06:31:48.142789+0000","last_undegraded":"2024-04-22T06:31:48.142789+0000","last_fullsized":"2024-04-22T06:31:48.142789+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:30:20.432210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142821+0000","last_change":"2024-04-22T06:31:07.022314+0000","last_active":"2024-04-22T06:31:48.142821+0000","last_peered":"2024-04-22T06:31:48.142821+0000","last_clean":"2024-04-22T06:31:48.142821+0000","last_became_active":"2024-04-22T06:31:07.021497+0000","last_became_peered":"2024-04-22T06:31:07.021497+0000","last_unstale":"2024-04-22T06:31:48.142821+0000","last_undegraded":"2024-04-22T06:31:48.142821+0000","last_fullsized":"2024-04-22T06:31:48.142821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:16:25.960359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'5","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:55.159169+0000","last_change":"2024-04-22T06:31:06.003805+0000","last_active":"2024-04-22T06:31:55.159169+0000","last_peered":"2024-04-22T06:31:55.159169+0000","last_clean":"2024-04-22T06:31:55.159169+0000","last_became_active":"2024-04-22T06:31:06.002735+0000","last_became_peered":"2024-04-22T06:31:06.002735+0000","last_unstale":"2024-04-22T06:31:55.159169+0000","last_undegraded":"2024-04-22T06:31:55.159169+0000","last_fullsized":"2024-04-22T06:31:55.159169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:41:16.155628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.141815+0000","last_change":"2024-04-22T06:31:11.043003+0000","last_active":"2024-04-22T06:31:48.141815+0000","last_peered":"2024-04-22T06:31:48.141815+0000","last_clean":"2024-04-22T06:31:48.141815+0000","last_became_active":"2024-04-22T06:31:11.042599+0000","last_became_peered":"2024-04-22T06:31:11.042599+0000","last_unstale":"2024-04-22T06:31:48.141815+0000","last_undegraded":"2024-04-22T06:31:48.141815+0000","last_fullsized":"2024-04-22T06:31:48.141815+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:28:58.949214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142697+0000","last_change":"2024-04-22T06:31:07.021258+0000","last_active":"2024-04-22T06:31:48.142697+0000","last_peered":"2024-04-22T06:31:48.142697+0000","last_clean":"2024-04-22T06:31:48.142697+0000","last_became_active":"2024-04-22T06:31:07.020976+0000","last_became_peered":"2024-04-22T06:31:07.020976+0000","last_unstale":"2024-04-22T06:31:48.142697+0000","last_undegraded":"2024-04-22T06:31:48.142697+0000","last_fullsized":"2024-04-22T06:31:48.142697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:19:33.377626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"27'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.191649+0000","last_change":"2024-04-22T06:31:06.002751+0000","last_active":"2024-04-22T06:31:48.191649+0000","last_peered":"2024-04-22T06:31:48.191649+0000","last_clean":"2024-04-22T06:31:48.191649+0000","last_became_active":"2024-04-22T06:31:06.002489+0000","last_became_peered":"2024-04-22T06:31:06.002489+0000","last_unstale":"2024-04-22T06:31:48.191649+0000","last_undegraded":"2024-04-22T06:31:48.191649+0000","last_fullsized":"2024-04-22T06:31:48.191649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:20:05.937529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.190750+0000","last_change":"2024-04-22T06:31:11.048954+0000","last_active":"2024-04-22T06:31:48.190750+0000","last_peered":"2024-04-22T06:31:48.190750+0000","last_clean":"2024-04-22T06:31:48.190750+0000","last_became_active":"2024-04-22T06:31:11.048214+0000","last_became_peered":"2024-04-22T06:31:11.048214+0000","last_unstale":"2024-04-22T06:31:48.190750+0000","last_undegraded":"2024-04-22T06:31:48.190750+0000","last_fullsized":"2024-04-22T06:31:48.190750+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:13:21.903203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142399+0000","last_change":"2024-04-22T06:31:07.021258+0000","last_active":"2024-04-22T06:31:48.142399+0000","last_peered":"2024-04-22T06:31:48.142399+0000","last_clean":"2024-04-22T06:31:48.142399+0000","last_became_active":"2024-04-22T06:31:07.021036+0000","last_became_peered":"2024-04-22T06:31:07.021036+0000","last_unstale":"2024-04-22T06:31:48.142399+0000","last_undegraded":"2024-04-22T06:31:48.142399+0000","last_fullsized":"2024-04-22T06:31:48.142399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:51:39.990228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'17","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:49.144742+0000","last_change":"2024-04-22T06:31:05.998270+0000","last_active":"2024-04-22T06:31:49.144742+0000","last_peered":"2024-04-22T06:31:49.144742+0000","last_clean":"2024-04-22T06:31:49.144742+0000","last_became_active":"2024-04-22T06:31:05.997925+0000","last_became_peered":"2024-04-22T06:31:05.997925+0000","last_unstale":"2024-04-22T06:31:49.144742+0000","last_undegraded":"2024-04-22T06:31:49.144742+0000","last_fullsized":"2024-04-22T06:31:49.144742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:49:44.167932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143553+0000","last_change":"2024-04-22T06:31:11.051106+0000","last_active":"2024-04-22T06:31:48.143553+0000","last_peered":"2024-04-22T06:31:48.143553+0000","last_clean":"2024-04-22T06:31:48.143553+0000","last_became_active":"2024-04-22T06:31:11.050721+0000","last_became_peered":"2024-04-22T06:31:11.050721+0000","last_unstale":"2024-04-22T06:31:48.143553+0000","last_undegraded":"2024-04-22T06:31:48.143553+0000","last_fullsized":"2024-04-22T06:31:48.143553+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:58:43.521850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145068+0000","last_change":"2024-04-22T06:31:07.020281+0000","last_active":"2024-04-22T06:31:48.145068+0000","last_peered":"2024-04-22T06:31:48.145068+0000","last_clean":"2024-04-22T06:31:48.145068+0000","last_became_active":"2024-04-22T06:31:07.020012+0000","last_became_peered":"2024-04-22T06:31:07.020012+0000","last_unstale":"2024-04-22T06:31:48.145068+0000","last_undegraded":"2024-04-22T06:31:48.145068+0000","last_fullsized":"2024-04-22T06:31:48.145068+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:20:40.301631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"27'5","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:49.139852+0000","last_change":"2024-04-22T06:31:05.999918+0000","last_active":"2024-04-22T06:31:49.139852+0000","last_peered":"2024-04-22T06:31:49.139852+0000","last_clean":"2024-04-22T06:31:49.139852+0000","last_became_active":"2024-04-22T06:31:05.999594+0000","last_became_peered":"2024-04-22T06:31:05.999594+0000","last_unstale":"2024-04-22T06:31:49.139852+0000","last_undegraded":"2024-04-22T06:31:49.139852+0000","last_fullsized":"2024-04-22T06:31:49.139852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:15:21.376196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.146252+0000","last_change":"2024-04-22T06:31:11.055940+0000","last_active":"2024-04-22T06:31:48.146252+0000","last_peered":"2024-04-22T06:31:48.146252+0000","last_clean":"2024-04-22T06:31:48.146252+0000","last_became_active":"2024-04-22T06:31:11.054873+0000","last_became_peered":"2024-04-22T06:31:11.054873+0000","last_unstale":"2024-04-22T06:31:48.146252+0000","last_undegraded":"2024-04-22T06:31:48.146252+0000","last_fullsized":"2024-04-22T06:31:48.146252+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:13:51.061199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143015+0000","last_change":"2024-04-22T06:31:07.021839+0000","last_active":"2024-04-22T06:31:48.143015+0000","last_peered":"2024-04-22T06:31:48.143015+0000","last_clean":"2024-04-22T06:31:48.143015+0000","last_became_active":"2024-04-22T06:31:07.021585+0000","last_became_peered":"2024-04-22T06:31:07.021585+0000","last_unstale":"2024-04-22T06:31:48.143015+0000","last_undegraded":"2024-04-22T06:31:48.143015+0000","last_fullsized":"2024-04-22T06:31:48.143015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:55:15.275802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"27'5","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:49.141566+0000","last_change":"2024-04-22T06:31:06.004058+0000","last_active":"2024-04-22T06:31:49.141566+0000","last_peered":"2024-04-22T06:31:49.141566+0000","last_clean":"2024-04-22T06:31:49.141566+0000","last_became_active":"2024-04-22T06:31:06.003791+0000","last_became_peered":"2024-04-22T06:31:06.003791+0000","last_unstale":"2024-04-22T06:31:49.141566+0000","last_undegraded":"2024-04-22T06:31:49.141566+0000","last_fullsized":"2024-04-22T06:31:49.141566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:03:08.913113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.146322+0000","last_change":"2024-04-22T06:31:11.054245+0000","last_active":"2024-04-22T06:31:48.146322+0000","last_peered":"2024-04-22T06:31:48.146322+0000","last_clean":"2024-04-22T06:31:48.146322+0000","last_became_active":"2024-04-22T06:31:11.053551+0000","last_became_peered":"2024-04-22T06:31:11.053551+0000","last_unstale":"2024-04-22T06:31:48.146322+0000","last_undegraded":"2024-04-22T06:31:48.146322+0000","last_fullsized":"2024-04-22T06:31:48.146322+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:34:06.312700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143337+0000","last_change":"2024-04-22T06:31:07.023273+0000","last_active":"2024-04-22T06:31:48.143337+0000","last_peered":"2024-04-22T06:31:48.143337+0000","last_clean":"2024-04-22T06:31:48.143337+0000","last_became_active":"2024-04-22T06:31:07.022978+0000","last_became_peered":"2024-04-22T06:31:07.022978+0000","last_unstale":"2024-04-22T06:31:48.143337+0000","last_undegraded":"2024-04-22T06:31:48.143337+0000","last_fullsized":"2024-04-22T06:31:48.143337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:10:50.192298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"27'9","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:55.160402+0000","last_change":"2024-04-22T06:31:06.005510+0000","last_active":"2024-04-22T06:31:55.160402+0000","last_peered":"2024-04-22T06:31:55.160402+0000","last_clean":"2024-04-22T06:31:55.160402+0000","last_became_active":"2024-04-22T06:31:06.005047+0000","last_became_peered":"2024-04-22T06:31:06.005047+0000","last_unstale":"2024-04-22T06:31:55.160402+0000","last_undegraded":"2024-04-22T06:31:55.160402+0000","last_fullsized":"2024-04-22T06:31:55.160402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:25:51.227631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143150+0000","last_change":"2024-04-22T06:31:11.046209+0000","last_active":"2024-04-22T06:31:48.143150+0000","last_peered":"2024-04-22T06:31:48.143150+0000","last_clean":"2024-04-22T06:31:48.143150+0000","last_became_active":"2024-04-22T06:31:11.045339+0000","last_became_peered":"2024-04-22T06:31:11.045339+0000","last_unstale":"2024-04-22T06:31:48.143150+0000","last_undegraded":"2024-04-22T06:31:48.143150+0000","last_fullsized":"2024-04-22T06:31:48.143150+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:39:16.659010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142146+0000","last_change":"2024-04-22T06:31:07.025901+0000","last_active":"2024-04-22T06:31:48.142146+0000","last_peered":"2024-04-22T06:31:48.142146+0000","last_clean":"2024-04-22T06:31:48.142146+0000","last_became_active":"2024-04-22T06:31:07.025590+0000","last_became_peered":"2024-04-22T06:31:07.025590+0000","last_unstale":"2024-04-22T06:31:48.142146+0000","last_undegraded":"2024-04-22T06:31:48.142146+0000","last_fullsized":"2024-04-22T06:31:48.142146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:00:26.184054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'13","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:55.161498+0000","last_change":"2024-04-22T06:31:06.004609+0000","last_active":"2024-04-22T06:31:55.161498+0000","last_peered":"2024-04-22T06:31:55.161498+0000","last_clean":"2024-04-22T06:31:55.161498+0000","last_became_active":"2024-04-22T06:31:06.004319+0000","last_became_peered":"2024-04-22T06:31:06.004319+0000","last_unstale":"2024-04-22T06:31:55.161498+0000","last_undegraded":"2024-04-22T06:31:55.161498+0000","last_fullsized":"2024-04-22T06:31:55.161498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:32:47.198906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.146760+0000","last_change":"2024-04-22T06:31:11.054706+0000","last_active":"2024-04-22T06:31:48.146760+0000","last_peered":"2024-04-22T06:31:48.146760+0000","last_clean":"2024-04-22T06:31:48.146760+0000","last_became_active":"2024-04-22T06:31:11.054307+0000","last_became_peered":"2024-04-22T06:31:11.054307+0000","last_unstale":"2024-04-22T06:31:48.146760+0000","last_undegraded":"2024-04-22T06:31:48.146760+0000","last_fullsized":"2024-04-22T06:31:48.146760+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:05:02.465114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.191527+0000","last_change":"2024-04-22T06:31:07.020739+0000","last_active":"2024-04-22T06:31:48.191527+0000","last_peered":"2024-04-22T06:31:48.191527+0000","last_clean":"2024-04-22T06:31:48.191527+0000","last_became_active":"2024-04-22T06:31:07.020070+0000","last_became_peered":"2024-04-22T06:31:07.020070+0000","last_unstale":"2024-04-22T06:31:48.191527+0000","last_undegraded":"2024-04-22T06:31:48.191527+0000","last_fullsized":"2024-04-22T06:31:48.191527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:50:22.102753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"27'5","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.147920+0000","last_change":"2024-04-22T06:31:06.003733+0000","last_active":"2024-04-22T06:31:48.147920+0000","last_peered":"2024-04-22T06:31:48.147920+0000","last_clean":"2024-04-22T06:31:48.147920+0000","last_became_active":"2024-04-22T06:31:06.003465+0000","last_became_peered":"2024-04-22T06:31:06.003465+0000","last_unstale":"2024-04-22T06:31:48.147920+0000","last_undegraded":"2024-04-22T06:31:48.147920+0000","last_fullsized":"2024-04-22T06:31:48.147920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:19:36.023311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.191203+0000","last_change":"2024-04-22T06:31:11.058041+0000","last_active":"2024-04-22T06:31:48.191203+0000","last_peered":"2024-04-22T06:31:48.191203+0000","last_clean":"2024-04-22T06:31:48.191203+0000","last_became_active":"2024-04-22T06:31:11.057243+0000","last_became_peered":"2024-04-22T06:31:11.057243+0000","last_unstale":"2024-04-22T06:31:48.191203+0000","last_undegraded":"2024-04-22T06:31:48.191203+0000","last_fullsized":"2024-04-22T06:31:48.191203+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:49:06.781140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142816+0000","last_change":"2024-04-22T06:31:07.015369+0000","last_active":"2024-04-22T06:31:48.142816+0000","last_peered":"2024-04-22T06:31:48.142816+0000","last_clean":"2024-04-22T06:31:48.142816+0000","last_became_active":"2024-04-22T06:31:07.015088+0000","last_became_peered":"2024-04-22T06:31:07.015088+0000","last_unstale":"2024-04-22T06:31:48.142816+0000","last_undegraded":"2024-04-22T06:31:48.142816+0000","last_fullsized":"2024-04-22T06:31:48.142816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:10:09.449507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"27'4","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:55.161244+0000","last_change":"2024-04-22T06:31:05.999636+0000","last_active":"2024-04-22T06:31:55.161244+0000","last_peered":"2024-04-22T06:31:55.161244+0000","last_clean":"2024-04-22T06:31:55.161244+0000","last_became_active":"2024-04-22T06:31:05.999329+0000","last_became_peered":"2024-04-22T06:31:05.999329+0000","last_unstale":"2024-04-22T06:31:55.161244+0000","last_undegraded":"2024-04-22T06:31:55.161244+0000","last_fullsized":"2024-04-22T06:31:55.161244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:09:06.670827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145230+0000","last_change":"2024-04-22T06:31:11.053530+0000","last_active":"2024-04-22T06:31:48.145230+0000","last_peered":"2024-04-22T06:31:48.145230+0000","last_clean":"2024-04-22T06:31:48.145230+0000","last_became_active":"2024-04-22T06:31:11.053100+0000","last_became_peered":"2024-04-22T06:31:11.053100+0000","last_unstale":"2024-04-22T06:31:48.145230+0000","last_undegraded":"2024-04-22T06:31:48.145230+0000","last_fullsized":"2024-04-22T06:31:48.145230+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:57:16.506840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142929+0000","last_change":"2024-04-22T06:31:07.015838+0000","last_active":"2024-04-22T06:31:48.142929+0000","last_peered":"2024-04-22T06:31:48.142929+0000","last_clean":"2024-04-22T06:31:48.142929+0000","last_became_active":"2024-04-22T06:31:07.015545+0000","last_became_peered":"2024-04-22T06:31:07.015545+0000","last_unstale":"2024-04-22T06:31:48.142929+0000","last_undegraded":"2024-04-22T06:31:48.142929+0000","last_fullsized":"2024-04-22T06:31:48.142929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:43:54.251061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"27'4","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145723+0000","last_change":"2024-04-22T06:31:06.005442+0000","last_active":"2024-04-22T06:31:48.145723+0000","last_peered":"2024-04-22T06:31:48.145723+0000","last_clean":"2024-04-22T06:31:48.145723+0000","last_became_active":"2024-04-22T06:31:06.005156+0000","last_became_peered":"2024-04-22T06:31:06.005156+0000","last_unstale":"2024-04-22T06:31:48.145723+0000","last_undegraded":"2024-04-22T06:31:48.145723+0000","last_fullsized":"2024-04-22T06:31:48.145723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:52:32.138519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145672+0000","last_change":"2024-04-22T06:31:11.046885+0000","last_active":"2024-04-22T06:31:48.145672+0000","last_peered":"2024-04-22T06:31:48.145672+0000","last_clean":"2024-04-22T06:31:48.145672+0000","last_became_active":"2024-04-22T06:31:11.046444+0000","last_became_peered":"2024-04-22T06:31:11.046444+0000","last_unstale":"2024-04-22T06:31:48.145672+0000","last_undegraded":"2024-04-22T06:31:48.145672+0000","last_fullsized":"2024-04-22T06:31:48.145672+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:17:54.523416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143477+0000","last_change":"2024-04-22T06:31:11.061784+0000","last_active":"2024-04-22T06:31:48.143477+0000","last_peered":"2024-04-22T06:31:48.143477+0000","last_clean":"2024-04-22T06:31:48.143477+0000","last_became_active":"2024-04-22T06:31:11.061382+0000","last_became_peered":"2024-04-22T06:31:11.061382+0000","last_unstale":"2024-04-22T06:31:48.143477+0000","last_undegraded":"2024-04-22T06:31:48.143477+0000","last_fullsized":"2024-04-22T06:31:48.143477+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:12:53.829206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.151807+0000","last_change":"2024-04-22T06:31:48.151807+0000","last_active":"2024-04-22T06:31:48.151807+0000","last_peered":"2024-04-22T06:31:48.151807+0000","last_clean":"2024-04-22T06:31:48.151807+0000","last_became_active":"2024-04-22T06:31:48.151525+0000","last_became_peered":"2024-04-22T06:31:48.151525+0000","last_unstale":"2024-04-22T06:31:48.151807+0000","last_undegraded":"2024-04-22T06:31:48.151807+0000","last_fullsized":"2024-04-22T06:31:48.151807+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:30:34.658957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"27'3","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:49.140808+0000","last_change":"2024-04-22T06:31:06.003080+0000","last_active":"2024-04-22T06:31:49.140808+0000","last_peered":"2024-04-22T06:31:49.140808+0000","last_clean":"2024-04-22T06:31:49.140808+0000","last_became_active":"2024-04-22T06:31:06.002725+0000","last_became_peered":"2024-04-22T06:31:06.002725+0000","last_unstale":"2024-04-22T06:31:49.140808+0000","last_undegraded":"2024-04-22T06:31:49.140808+0000","last_fullsized":"2024-04-22T06:31:49.140808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:46:38.249137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145623+0000","last_change":"2024-04-22T06:31:11.047782+0000","last_active":"2024-04-22T06:31:48.145623+0000","last_peered":"2024-04-22T06:31:48.145623+0000","last_clean":"2024-04-22T06:31:48.145623+0000","last_became_active":"2024-04-22T06:31:11.047407+0000","last_became_peered":"2024-04-22T06:31:11.047407+0000","last_unstale":"2024-04-22T06:31:48.145623+0000","last_undegraded":"2024-04-22T06:31:48.145623+0000","last_fullsized":"2024-04-22T06:31:48.145623+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:48:05.416994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145318+0000","last_change":"2024-04-22T06:31:07.025799+0000","last_active":"2024-04-22T06:31:48.145318+0000","last_peered":"2024-04-22T06:31:48.145318+0000","last_clean":"2024-04-22T06:31:48.145318+0000","last_became_active":"2024-04-22T06:31:07.025420+0000","last_became_peered":"2024-04-22T06:31:07.025420+0000","last_unstale":"2024-04-22T06:31:48.145318+0000","last_undegraded":"2024-04-22T06:31:48.145318+0000","last_fullsized":"2024-04-22T06:31:48.145318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:39:40.107621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"27'3","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.146171+0000","last_change":"2024-04-22T06:31:06.005805+0000","last_active":"2024-04-22T06:31:48.146171+0000","last_peered":"2024-04-22T06:31:48.146171+0000","last_clean":"2024-04-22T06:31:48.146171+0000","last_became_active":"2024-04-22T06:31:06.005479+0000","last_became_peered":"2024-04-22T06:31:06.005479+0000","last_unstale":"2024-04-22T06:31:48.146171+0000","last_undegraded":"2024-04-22T06:31:48.146171+0000","last_fullsized":"2024-04-22T06:31:48.146171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:59:37.140612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"27'7","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:49.140833+0000","last_change":"2024-04-22T06:31:05.998354+0000","last_active":"2024-04-22T06:31:49.140833+0000","last_peered":"2024-04-22T06:31:49.140833+0000","last_clean":"2024-04-22T06:31:49.140833+0000","last_became_active":"2024-04-22T06:31:05.998071+0000","last_became_peered":"2024-04-22T06:31:05.998071+0000","last_unstale":"2024-04-22T06:31:49.140833+0000","last_undegraded":"2024-04-22T06:31:49.140833+0000","last_fullsized":"2024-04-22T06:31:49.140833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:12:13.343435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142976+0000","last_change":"2024-04-22T06:31:07.024109+0000","last_active":"2024-04-22T06:31:48.142976+0000","last_peered":"2024-04-22T06:31:48.142976+0000","last_clean":"2024-04-22T06:31:48.142976+0000","last_became_active":"2024-04-22T06:31:07.023346+0000","last_became_peered":"2024-04-22T06:31:07.023346+0000","last_unstale":"2024-04-22T06:31:48.142976+0000","last_undegraded":"2024-04-22T06:31:48.142976+0000","last_fullsized":"2024-04-22T06:31:48.142976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:02:42.662312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145564+0000","last_change":"2024-04-22T06:31:11.047132+0000","last_active":"2024-04-22T06:31:48.145564+0000","last_peered":"2024-04-22T06:31:48.145564+0000","last_clean":"2024-04-22T06:31:48.145564+0000","last_became_active":"2024-04-22T06:31:11.046671+0000","last_became_peered":"2024-04-22T06:31:11.046671+0000","last_unstale":"2024-04-22T06:31:48.145564+0000","last_undegraded":"2024-04-22T06:31:48.145564+0000","last_fullsized":"2024-04-22T06:31:48.145564+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:41:57.706735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"25'5","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.146032+0000","last_change":"2024-04-22T06:31:06.004971+0000","last_active":"2024-04-22T06:31:48.146032+0000","last_peered":"2024-04-22T06:31:48.146032+0000","last_clean":"2024-04-22T06:31:48.146032+0000","last_became_active":"2024-04-22T06:31:06.004473+0000","last_became_peered":"2024-04-22T06:31:06.004473+0000","last_unstale":"2024-04-22T06:31:48.146032+0000","last_undegraded":"2024-04-22T06:31:48.146032+0000","last_fullsized":"2024-04-22T06:31:48.146032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:47:11.735311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.191136+0000","last_change":"2024-04-22T06:31:07.018314+0000","last_active":"2024-04-22T06:31:48.191136+0000","last_peered":"2024-04-22T06:31:48.191136+0000","last_clean":"2024-04-22T06:31:48.191136+0000","last_became_active":"2024-04-22T06:31:07.018033+0000","last_became_peered":"2024-04-22T06:31:07.018033+0000","last_unstale":"2024-04-22T06:31:48.191136+0000","last_undegraded":"2024-04-22T06:31:48.191136+0000","last_fullsized":"2024-04-22T06:31:48.191136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:06:06.099494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.192586+0000","last_change":"2024-04-22T06:31:48.192586+0000","last_active":"2024-04-22T06:31:48.192586+0000","last_peered":"2024-04-22T06:31:48.192586+0000","last_clean":"2024-04-22T06:31:48.192586+0000","last_became_active":"2024-04-22T06:31:48.192127+0000","last_became_peered":"2024-04-22T06:31:48.192127+0000","last_unstale":"2024-04-22T06:31:48.192586+0000","last_undegraded":"2024-04-22T06:31:48.192586+0000","last_fullsized":"2024-04-22T06:31:48.192586+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:58:18.053308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4,2],"acting":[3,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.144952+0000","last_change":"2024-04-22T06:31:07.024780+0000","last_active":"2024-04-22T06:31:48.144952+0000","last_peered":"2024-04-22T06:31:48.144952+0000","last_clean":"2024-04-22T06:31:48.144952+0000","last_became_active":"2024-04-22T06:31:07.024501+0000","last_became_peered":"2024-04-22T06:31:07.024501+0000","last_unstale":"2024-04-22T06:31:48.144952+0000","last_undegraded":"2024-04-22T06:31:48.144952+0000","last_fullsized":"2024-04-22T06:31:48.144952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:12:45.981232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.150411+0000","last_change":"2024-04-22T06:31:48.150411+0000","last_active":"2024-04-22T06:31:48.150411+0000","last_peered":"2024-04-22T06:31:48.150411+0000","last_clean":"2024-04-22T06:31:48.150411+0000","last_became_active":"2024-04-22T06:31:48.150131+0000","last_became_peered":"2024-04-22T06:31:48.150131+0000","last_unstale":"2024-04-22T06:31:48.150411+0000","last_undegraded":"2024-04-22T06:31:48.150411+0000","last_fullsized":"2024-04-22T06:31:48.150411+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:32:05.456771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143270+0000","last_change":"2024-04-22T06:31:07.018563+0000","last_active":"2024-04-22T06:31:48.143270+0000","last_peered":"2024-04-22T06:31:48.143270+0000","last_clean":"2024-04-22T06:31:48.143270+0000","last_became_active":"2024-04-22T06:31:07.018289+0000","last_became_peered":"2024-04-22T06:31:07.018289+0000","last_unstale":"2024-04-22T06:31:48.143270+0000","last_undegraded":"2024-04-22T06:31:48.143270+0000","last_fullsized":"2024-04-22T06:31:48.143270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:30:11.892544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142888+0000","last_change":"2024-04-22T06:31:07.022901+0000","last_active":"2024-04-22T06:31:48.142888+0000","last_peered":"2024-04-22T06:31:48.142888+0000","last_clean":"2024-04-22T06:31:48.142888+0000","last_became_active":"2024-04-22T06:31:07.022420+0000","last_became_peered":"2024-04-22T06:31:07.022420+0000","last_unstale":"2024-04-22T06:31:48.142888+0000","last_undegraded":"2024-04-22T06:31:48.142888+0000","last_fullsized":"2024-04-22T06:31:48.142888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:24:18.929896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.146584+0000","last_change":"2024-04-22T06:31:07.022875+0000","last_active":"2024-04-22T06:31:48.146584+0000","last_peered":"2024-04-22T06:31:48.146584+0000","last_clean":"2024-04-22T06:31:48.146584+0000","last_became_active":"2024-04-22T06:31:07.022545+0000","last_became_peered":"2024-04-22T06:31:07.022545+0000","last_unstale":"2024-04-22T06:31:48.146584+0000","last_undegraded":"2024-04-22T06:31:48.146584+0000","last_fullsized":"2024-04-22T06:31:48.146584+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:02:30.464508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.147729+0000","last_change":"2024-04-22T06:31:07.026367+0000","last_active":"2024-04-22T06:31:48.147729+0000","last_peered":"2024-04-22T06:31:48.147729+0000","last_clean":"2024-04-22T06:31:48.147729+0000","last_became_active":"2024-04-22T06:31:07.026085+0000","last_became_peered":"2024-04-22T06:31:07.026085+0000","last_unstale":"2024-04-22T06:31:48.147729+0000","last_undegraded":"2024-04-22T06:31:48.147729+0000","last_fullsized":"2024-04-22T06:31:48.147729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:02:55.368591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145845+0000","last_change":"2024-04-22T06:31:07.022989+0000","last_active":"2024-04-22T06:31:48.145845+0000","last_peered":"2024-04-22T06:31:48.145845+0000","last_clean":"2024-04-22T06:31:48.145845+0000","last_became_active":"2024-04-22T06:31:07.022688+0000","last_became_peered":"2024-04-22T06:31:07.022688+0000","last_unstale":"2024-04-22T06:31:48.145845+0000","last_undegraded":"2024-04-22T06:31:48.145845+0000","last_fullsized":"2024-04-22T06:31:48.145845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:24:10.910731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.146336+0000","last_change":"2024-04-22T06:31:07.026315+0000","last_active":"2024-04-22T06:31:48.146336+0000","last_peered":"2024-04-22T06:31:48.146336+0000","last_clean":"2024-04-22T06:31:48.146336+0000","last_became_active":"2024-04-22T06:31:07.025951+0000","last_became_peered":"2024-04-22T06:31:07.025951+0000","last_unstale":"2024-04-22T06:31:48.146336+0000","last_undegraded":"2024-04-22T06:31:48.146336+0000","last_fullsized":"2024-04-22T06:31:48.146336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:54:12.160058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142093+0000","last_change":"2024-04-22T06:31:07.022748+0000","last_active":"2024-04-22T06:31:48.142093+0000","last_peered":"2024-04-22T06:31:48.142093+0000","last_clean":"2024-04-22T06:31:48.142093+0000","last_became_active":"2024-04-22T06:31:07.022481+0000","last_became_peered":"2024-04-22T06:31:07.022481+0000","last_unstale":"2024-04-22T06:31:48.142093+0000","last_undegraded":"2024-04-22T06:31:48.142093+0000","last_fullsized":"2024-04-22T06:31:48.142093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:56:32.631637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145948+0000","last_change":"2024-04-22T06:31:07.021939+0000","last_active":"2024-04-22T06:31:48.145948+0000","last_peered":"2024-04-22T06:31:48.145948+0000","last_clean":"2024-04-22T06:31:48.145948+0000","last_became_active":"2024-04-22T06:31:07.021638+0000","last_became_peered":"2024-04-22T06:31:07.021638+0000","last_unstale":"2024-04-22T06:31:48.145948+0000","last_undegraded":"2024-04-22T06:31:48.145948+0000","last_fullsized":"2024-04-22T06:31:48.145948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:05:27.106092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143165+0000","last_change":"2024-04-22T06:31:07.015844+0000","last_active":"2024-04-22T06:31:48.143165+0000","last_peered":"2024-04-22T06:31:48.143165+0000","last_clean":"2024-04-22T06:31:48.143165+0000","last_became_active":"2024-04-22T06:31:07.015520+0000","last_became_peered":"2024-04-22T06:31:07.015520+0000","last_unstale":"2024-04-22T06:31:48.143165+0000","last_undegraded":"2024-04-22T06:31:48.143165+0000","last_fullsized":"2024-04-22T06:31:48.143165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:24:04.676892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142694+0000","last_change":"2024-04-22T06:31:07.023697+0000","last_active":"2024-04-22T06:31:48.142694+0000","last_peered":"2024-04-22T06:31:48.142694+0000","last_clean":"2024-04-22T06:31:48.142694+0000","last_became_active":"2024-04-22T06:31:07.023283+0000","last_became_peered":"2024-04-22T06:31:07.023283+0000","last_unstale":"2024-04-22T06:31:48.142694+0000","last_undegraded":"2024-04-22T06:31:48.142694+0000","last_fullsized":"2024-04-22T06:31:48.142694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:43:01.338104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145399+0000","last_change":"2024-04-22T06:31:07.023953+0000","last_active":"2024-04-22T06:31:48.145399+0000","last_peered":"2024-04-22T06:31:48.145399+0000","last_clean":"2024-04-22T06:31:48.145399+0000","last_became_active":"2024-04-22T06:31:07.023704+0000","last_became_peered":"2024-04-22T06:31:07.023704+0000","last_unstale":"2024-04-22T06:31:48.145399+0000","last_undegraded":"2024-04-22T06:31:48.145399+0000","last_fullsized":"2024-04-22T06:31:48.145399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:59:32.219492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142410+0000","last_change":"2024-04-22T06:31:07.027418+0000","last_active":"2024-04-22T06:31:48.142410+0000","last_peered":"2024-04-22T06:31:48.142410+0000","last_clean":"2024-04-22T06:31:48.142410+0000","last_became_active":"2024-04-22T06:31:07.027113+0000","last_became_peered":"2024-04-22T06:31:07.027113+0000","last_unstale":"2024-04-22T06:31:48.142410+0000","last_undegraded":"2024-04-22T06:31:48.142410+0000","last_fullsized":"2024-04-22T06:31:48.142410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:08:42.204674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145946+0000","last_change":"2024-04-22T06:31:07.015520+0000","last_active":"2024-04-22T06:31:48.145946+0000","last_peered":"2024-04-22T06:31:48.145946+0000","last_clean":"2024-04-22T06:31:48.145946+0000","last_became_active":"2024-04-22T06:31:07.015199+0000","last_became_peered":"2024-04-22T06:31:07.015199+0000","last_unstale":"2024-04-22T06:31:48.145946+0000","last_undegraded":"2024-04-22T06:31:48.145946+0000","last_fullsized":"2024-04-22T06:31:48.145946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:48:40.688931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142835+0000","last_change":"2024-04-22T06:31:07.016457+0000","last_active":"2024-04-22T06:31:48.142835+0000","last_peered":"2024-04-22T06:31:48.142835+0000","last_clean":"2024-04-22T06:31:48.142835+0000","last_became_active":"2024-04-22T06:31:07.016082+0000","last_became_peered":"2024-04-22T06:31:07.016082+0000","last_unstale":"2024-04-22T06:31:48.142835+0000","last_undegraded":"2024-04-22T06:31:48.142835+0000","last_fullsized":"2024-04-22T06:31:48.142835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:12:28.517065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.141985+0000","last_change":"2024-04-22T06:31:07.019955+0000","last_active":"2024-04-22T06:31:48.141985+0000","last_peered":"2024-04-22T06:31:48.141985+0000","last_clean":"2024-04-22T06:31:48.141985+0000","last_became_active":"2024-04-22T06:31:07.019174+0000","last_became_peered":"2024-04-22T06:31:07.019174+0000","last_unstale":"2024-04-22T06:31:48.141985+0000","last_undegraded":"2024-04-22T06:31:48.141985+0000","last_fullsized":"2024-04-22T06:31:48.141985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:09:07.106161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.147563+0000","last_change":"2024-04-22T06:31:07.019341+0000","last_active":"2024-04-22T06:31:48.147563+0000","last_peered":"2024-04-22T06:31:48.147563+0000","last_clean":"2024-04-22T06:31:48.147563+0000","last_became_active":"2024-04-22T06:31:07.019048+0000","last_became_peered":"2024-04-22T06:31:07.019048+0000","last_unstale":"2024-04-22T06:31:48.147563+0000","last_undegraded":"2024-04-22T06:31:48.147563+0000","last_fullsized":"2024-04-22T06:31:48.147563+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:04:19.362686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143194+0000","last_change":"2024-04-22T06:31:07.019433+0000","last_active":"2024-04-22T06:31:48.143194+0000","last_peered":"2024-04-22T06:31:48.143194+0000","last_clean":"2024-04-22T06:31:48.143194+0000","last_became_active":"2024-04-22T06:31:07.019133+0000","last_became_peered":"2024-04-22T06:31:07.019133+0000","last_unstale":"2024-04-22T06:31:48.143194+0000","last_undegraded":"2024-04-22T06:31:48.143194+0000","last_fullsized":"2024-04-22T06:31:48.143194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:01:54.466486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145847+0000","last_change":"2024-04-22T06:31:07.023014+0000","last_active":"2024-04-22T06:31:48.145847+0000","last_peered":"2024-04-22T06:31:48.145847+0000","last_clean":"2024-04-22T06:31:48.145847+0000","last_became_active":"2024-04-22T06:31:07.022716+0000","last_became_peered":"2024-04-22T06:31:07.022716+0000","last_unstale":"2024-04-22T06:31:48.145847+0000","last_undegraded":"2024-04-22T06:31:48.145847+0000","last_fullsized":"2024-04-22T06:31:48.145847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:11:47.309706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.191327+0000","last_change":"2024-04-22T06:31:07.020446+0000","last_active":"2024-04-22T06:31:48.191327+0000","last_peered":"2024-04-22T06:31:48.191327+0000","last_clean":"2024-04-22T06:31:48.191327+0000","last_became_active":"2024-04-22T06:31:07.018995+0000","last_became_peered":"2024-04-22T06:31:07.018995+0000","last_unstale":"2024-04-22T06:31:48.191327+0000","last_undegraded":"2024-04-22T06:31:48.191327+0000","last_fullsized":"2024-04-22T06:31:48.191327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:56:45.031262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.149688+0000","last_change":"2024-04-22T06:31:48.149688+0000","last_active":"2024-04-22T06:31:48.149688+0000","last_peered":"2024-04-22T06:31:48.149688+0000","last_clean":"2024-04-22T06:31:48.149688+0000","last_became_active":"2024-04-22T06:31:48.149398+0000","last_became_peered":"2024-04-22T06:31:48.149398+0000","last_unstale":"2024-04-22T06:31:48.149688+0000","last_undegraded":"2024-04-22T06:31:48.149688+0000","last_fullsized":"2024-04-22T06:31:48.149688+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:55:49.846070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145780+0000","last_change":"2024-04-22T06:31:07.014209+0000","last_active":"2024-04-22T06:31:48.145780+0000","last_peered":"2024-04-22T06:31:48.145780+0000","last_clean":"2024-04-22T06:31:48.145780+0000","last_became_active":"2024-04-22T06:31:07.013946+0000","last_became_peered":"2024-04-22T06:31:07.013946+0000","last_unstale":"2024-04-22T06:31:48.145780+0000","last_undegraded":"2024-04-22T06:31:48.145780+0000","last_fullsized":"2024-04-22T06:31:48.145780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:40:31.669158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143602+0000","last_change":"2024-04-22T06:31:07.008764+0000","last_active":"2024-04-22T06:31:48.143602+0000","last_peered":"2024-04-22T06:31:48.143602+0000","last_clean":"2024-04-22T06:31:48.143602+0000","last_became_active":"2024-04-22T06:31:07.007810+0000","last_became_peered":"2024-04-22T06:31:07.007810+0000","last_unstale":"2024-04-22T06:31:48.143602+0000","last_undegraded":"2024-04-22T06:31:48.143602+0000","last_fullsized":"2024-04-22T06:31:48.143602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:49:54.894818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145136+0000","last_change":"2024-04-22T06:31:07.022141+0000","last_active":"2024-04-22T06:31:48.145136+0000","last_peered":"2024-04-22T06:31:48.145136+0000","last_clean":"2024-04-22T06:31:48.145136+0000","last_became_active":"2024-04-22T06:31:07.021845+0000","last_became_peered":"2024-04-22T06:31:07.021845+0000","last_unstale":"2024-04-22T06:31:48.145136+0000","last_undegraded":"2024-04-22T06:31:48.145136+0000","last_fullsized":"2024-04-22T06:31:48.145136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:26:28.598104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142294+0000","last_change":"2024-04-22T06:31:07.012188+0000","last_active":"2024-04-22T06:31:48.142294+0000","last_peered":"2024-04-22T06:31:48.142294+0000","last_clean":"2024-04-22T06:31:48.142294+0000","last_became_active":"2024-04-22T06:31:07.011909+0000","last_became_peered":"2024-04-22T06:31:07.011909+0000","last_unstale":"2024-04-22T06:31:48.142294+0000","last_undegraded":"2024-04-22T06:31:48.142294+0000","last_fullsized":"2024-04-22T06:31:48.142294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:25:13.089928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143079+0000","last_change":"2024-04-22T06:31:07.020033+0000","last_active":"2024-04-22T06:31:48.143079+0000","last_peered":"2024-04-22T06:31:48.143079+0000","last_clean":"2024-04-22T06:31:48.143079+0000","last_became_active":"2024-04-22T06:31:07.019689+0000","last_became_peered":"2024-04-22T06:31:07.019689+0000","last_unstale":"2024-04-22T06:31:48.143079+0000","last_undegraded":"2024-04-22T06:31:48.143079+0000","last_fullsized":"2024-04-22T06:31:48.143079+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:09:07.209203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142885+0000","last_change":"2024-04-22T06:31:07.027287+0000","last_active":"2024-04-22T06:31:48.142885+0000","last_peered":"2024-04-22T06:31:48.142885+0000","last_clean":"2024-04-22T06:31:48.142885+0000","last_became_active":"2024-04-22T06:31:07.027015+0000","last_became_peered":"2024-04-22T06:31:07.027015+0000","last_unstale":"2024-04-22T06:31:48.142885+0000","last_undegraded":"2024-04-22T06:31:48.142885+0000","last_fullsized":"2024-04-22T06:31:48.142885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:42:49.322413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143284+0000","last_change":"2024-04-22T06:31:07.026332+0000","last_active":"2024-04-22T06:31:48.143284+0000","last_peered":"2024-04-22T06:31:48.143284+0000","last_clean":"2024-04-22T06:31:48.143284+0000","last_became_active":"2024-04-22T06:31:07.026018+0000","last_became_peered":"2024-04-22T06:31:07.026018+0000","last_unstale":"2024-04-22T06:31:48.143284+0000","last_undegraded":"2024-04-22T06:31:48.143284+0000","last_fullsized":"2024-04-22T06:31:48.143284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:27:08.560056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143261+0000","last_change":"2024-04-22T06:31:07.020497+0000","last_active":"2024-04-22T06:31:48.143261+0000","last_peered":"2024-04-22T06:31:48.143261+0000","last_clean":"2024-04-22T06:31:48.143261+0000","last_became_active":"2024-04-22T06:31:07.020194+0000","last_became_peered":"2024-04-22T06:31:07.020194+0000","last_unstale":"2024-04-22T06:31:48.143261+0000","last_undegraded":"2024-04-22T06:31:48.143261+0000","last_fullsized":"2024-04-22T06:31:48.143261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:56:59.765295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.148064+0000","last_change":"2024-04-22T06:31:07.018396+0000","last_active":"2024-04-22T06:31:48.148064+0000","last_peered":"2024-04-22T06:31:48.148064+0000","last_clean":"2024-04-22T06:31:48.148064+0000","last_became_active":"2024-04-22T06:31:07.018073+0000","last_became_peered":"2024-04-22T06:31:07.018073+0000","last_unstale":"2024-04-22T06:31:48.148064+0000","last_undegraded":"2024-04-22T06:31:48.148064+0000","last_fullsized":"2024-04-22T06:31:48.148064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:37:06.501499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142711+0000","last_change":"2024-04-22T06:31:07.014439+0000","last_active":"2024-04-22T06:31:48.142711+0000","last_peered":"2024-04-22T06:31:48.142711+0000","last_clean":"2024-04-22T06:31:48.142711+0000","last_became_active":"2024-04-22T06:31:07.014124+0000","last_became_peered":"2024-04-22T06:31:07.014124+0000","last_unstale":"2024-04-22T06:31:48.142711+0000","last_undegraded":"2024-04-22T06:31:48.142711+0000","last_fullsized":"2024-04-22T06:31:48.142711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:58:46.075068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":10264,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":208,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":20528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":182,"ondisk_log_size":182,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32076,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339764,"statfs":{"total":96636764160,"available":96603918336,"internally_reserved":0,"allocated":921600,"data_stored":711508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31488,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340352,"statfs":{"total":96636764160,"available":96604520448,"internally_reserved":0,"allocated":319488,"data_stored":123672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":339968,"data_stored":122972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31484,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340356,"statfs":{"total":96636764160,"available":96604524544,"internally_reserved":0,"allocated":315392,"data_stored":120370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":323584,"data_stored":123706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":323584,"data_stored":120502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":319488,"data_stored":120572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343932,"statfs":{"total":96636764160,"available":96608186368,"internally_reserved":0,"allocated":913408,"data_stored":710282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T06:32:07.763 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T06:32:07.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.900+0000 7fde5f4d2640 1 Processor -- start 2024-04-22T06:32:07.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.901+0000 7fde5f4d2640 1 -- start start 2024-04-22T06:32:07.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.901+0000 7fde5f4d2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fde5815e670 0x7fde5815ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:07.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.901+0000 7fde5f4d2640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fde580cd170 0x7fde580cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:07.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.901+0000 7fde5f4d2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde580cb990 con 0x7fde580c7d00 2024-04-22T06:32:07.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.901+0000 7fde5f4d2640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fde5815f480 con 0x7fde580cd170 2024-04-22T06:32:07.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.901+0000 7fde5f4d2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fde581558d0 con 0x7fde5815e670 2024-04-22T06:32:07.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.901+0000 7fde5da6f640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fde580cd170 0x7fde580cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:07.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.902+0000 7fde5da6f640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fde580cd170 0x7fde580cd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52496/0 (socket says 172.21.15.44:52496) 2024-04-22T06:32:07.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.902+0000 7fde5da6f640 1 -- 172.21.15.44:0/1932624899 learned_addr learned my addr 172.21.15.44:0/1932624899 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:07.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.902+0000 7fde5d26e640 1 --2- 172.21.15.44:0/1932624899 >> v2:172.21.15.70:3300/0 conn(0x7fde5815e670 0x7fde5815ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:07.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.902+0000 7fde5da6f640 1 -- 172.21.15.44:0/1932624899 >> v2:172.21.15.70:3300/0 conn(0x7fde5815e670 msgr2=0x7fde5815ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:07.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.902+0000 7fde5da6f640 1 --2- 172.21.15.44:0/1932624899 >> v2:172.21.15.70:3300/0 conn(0x7fde5815e670 0x7fde5815ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:07.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.902+0000 7fde5da6f640 1 -- 172.21.15.44:0/1932624899 >> v1:172.21.15.70:6789/0 conn(0x7fde580c7d00 legacy=0x7fde5806d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:07.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.902+0000 7fde5da6f640 1 -- 172.21.15.44:0/1932624899 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde580cc530 con 0x7fde580cd170 2024-04-22T06:32:07.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.902+0000 7fde5d26e640 1 --2- 172.21.15.44:0/1932624899 >> v2:172.21.15.70:3300/0 conn(0x7fde5815e670 0x7fde5815ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:32:07.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.903+0000 7fde5da6f640 1 --2- 172.21.15.44:0/1932624899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde580cd170 0x7fde580cd580 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fde580cb960 tx=0x7fde5002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=963e9ac5774e333f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:07.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.903+0000 7fde4ffff640 1 -- 172.21.15.44:0/1932624899 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5002ca80 con 0x7fde580cd170 2024-04-22T06:32:07.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.903+0000 7fde4ffff640 1 -- 172.21.15.44:0/1932624899 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde5002cc20 con 0x7fde580cd170 2024-04-22T06:32:07.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.903+0000 7fde4ffff640 1 -- 172.21.15.44:0/1932624899 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde500343f0 con 0x7fde580cd170 2024-04-22T06:32:07.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.903+0000 7fde5f4d2640 1 -- 172.21.15.44:0/1932624899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde580cd170 msgr2=0x7fde580cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:07.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.903+0000 7fde5f4d2640 1 --2- 172.21.15.44:0/1932624899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde580cd170 0x7fde580cd580 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fde580cb960 tx=0x7fde5002bee0 comp rx=0 tx=0).stop 2024-04-22T06:32:07.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.903+0000 7fde5f4d2640 1 -- 172.21.15.44:0/1932624899 shutdown_connections 2024-04-22T06:32:07.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.903+0000 7fde5f4d2640 1 --2- 172.21.15.44:0/1932624899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde580cd170 0x7fde580cd580 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:07.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.903+0000 7fde5f4d2640 1 --2- 172.21.15.44:0/1932624899 >> v2:172.21.15.70:3300/0 conn(0x7fde5815e670 0x7fde5815ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:07.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.903+0000 7fde5f4d2640 1 -- 172.21.15.44:0/1932624899 >> 172.21.15.44:0/1932624899 conn(0x7fde58072fb0 msgr2=0x7fde580733c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:07.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.903+0000 7fde5f4d2640 1 -- 172.21.15.44:0/1932624899 shutdown_connections 2024-04-22T06:32:07.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.903+0000 7fde5f4d2640 1 -- 172.21.15.44:0/1932624899 wait complete. 2024-04-22T06:32:07.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.905+0000 7fde5f4d2640 1 Processor -- start 2024-04-22T06:32:07.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.905+0000 7fde5f4d2640 1 -- start start 2024-04-22T06:32:07.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.905+0000 7fde5f4d2640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde580c7d00 0x7fde581563f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:07.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.905+0000 7fde5f4d2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde580cd170 0x7fde581e7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:07.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.905+0000 7fde5f4d2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde5815e670 0x7fde58156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:07.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.905+0000 7fde5f4d2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fde580ccaf0 con 0x7fde580cd170 2024-04-22T06:32:07.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.905+0000 7fde5f4d2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fde580cbf70 con 0x7fde5815e670 2024-04-22T06:32:07.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.905+0000 7fde5f4d2640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fde580cbc00 con 0x7fde580c7d00 2024-04-22T06:32:07.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.905+0000 7fde5ca6d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde580cd170 0x7fde581e7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:07.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.905+0000 7fde5ca6d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde580cd170 0x7fde581e7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52508/0 (socket says 172.21.15.44:52508) 2024-04-22T06:32:07.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.905+0000 7fde5ca6d640 1 -- 172.21.15.44:0/2853292990 learned_addr learned my addr 172.21.15.44:0/2853292990 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:07.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.906+0000 7fde5d26e640 1 --2- 172.21.15.44:0/2853292990 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde580c7d00 0x7fde581563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:07.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.906+0000 7fde5da6f640 1 --2- 172.21.15.44:0/2853292990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde5815e670 0x7fde58156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:07.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.906+0000 7fde5ca6d640 1 -- 172.21.15.44:0/2853292990 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde580c7d00 msgr2=0x7fde581563f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:07.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.906+0000 7fde5ca6d640 1 --2- 172.21.15.44:0/2853292990 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde580c7d00 0x7fde581563f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:07.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.906+0000 7fde5ca6d640 1 -- 172.21.15.44:0/2853292990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde5815e670 msgr2=0x7fde58156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:07.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.906+0000 7fde5ca6d640 1 --2- 172.21.15.44:0/2853292990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde5815e670 0x7fde58156d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:07.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.906+0000 7fde5ca6d640 1 -- 172.21.15.44:0/2853292990 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde581f10a0 con 0x7fde580cd170 2024-04-22T06:32:07.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.906+0000 7fde5ca6d640 1 --2- 172.21.15.44:0/2853292990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde580cd170 0x7fde581e7f30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fde4800c400 tx=0x7fde4800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:07.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.906+0000 7fde4dffb640 1 -- 172.21.15.44:0/2853292990 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde480052d0 con 0x7fde580cd170 2024-04-22T06:32:07.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.906+0000 7fde4dffb640 1 -- 172.21.15.44:0/2853292990 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde48005490 con 0x7fde580cd170 2024-04-22T06:32:07.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.906+0000 7fde4dffb640 1 -- 172.21.15.44:0/2853292990 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde48007260 con 0x7fde580cd170 2024-04-22T06:32:07.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.906+0000 7fde5f4d2640 1 -- 172.21.15.44:0/2853292990 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde581f4dc0 con 0x7fde580cd170 2024-04-22T06:32:07.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.906+0000 7fde5f4d2640 1 -- 172.21.15.44:0/2853292990 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde581f1340 con 0x7fde580cd170 2024-04-22T06:32:07.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.907+0000 7fde5f4d2640 1 -- 172.21.15.44:0/2853292990 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde581f0e90 con 0x7fde580cd170 2024-04-22T06:32:07.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.908+0000 7fde4dffb640 1 -- 172.21.15.44:0/2853292990 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fde48007400 con 0x7fde580cd170 2024-04-22T06:32:07.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.908+0000 7fde4dffb640 1 --2- 172.21.15.44:0/2853292990 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fde3c06c8f0 0x7fde3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:07.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.908+0000 7fde4dffb640 1 -- 172.21.15.44:0/2853292990 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fde4808b3f0 con 0x7fde580cd170 2024-04-22T06:32:07.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.908+0000 7fde5d26e640 1 --2- 172.21.15.44:0/2853292990 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fde3c06c8f0 0x7fde3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:07.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.909+0000 7fde5d26e640 1 --2- 172.21.15.44:0/2853292990 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fde3c06c8f0 0x7fde3c06edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fde54006fd0 tx=0x7fde54008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:07.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:07.912+0000 7fde4dffb640 1 -- 172.21.15.44:0/2853292990 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fde480594b0 con 0x7fde580cd170 2024-04-22T06:32:08.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.095+0000 7fde5f4d2640 1 -- 172.21.15.44:0/2853292990 --> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fde58156930 con 0x7fde3c06c8f0 2024-04-22T06:32:08.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.103+0000 7fde4dffb640 1 -- 172.21.15.44:0/2853292990 <== mgr.4111 v2:172.21.15.70:6832/2570466131 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332632 (secure 0 0 0) 0x7fde58156930 con 0x7fde3c06c8f0 2024-04-22T06:32:08.105 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:32:08.107 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-04-22T06:32:08.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.106+0000 7fde5f4d2640 1 -- 172.21.15.44:0/2853292990 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fde3c06c8f0 msgr2=0x7fde3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:08.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.106+0000 7fde5f4d2640 1 --2- 172.21.15.44:0/2853292990 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fde3c06c8f0 0x7fde3c06edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fde54006fd0 tx=0x7fde54008040 comp rx=0 tx=0).stop 2024-04-22T06:32:08.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.106+0000 7fde5f4d2640 1 -- 172.21.15.44:0/2853292990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde580cd170 msgr2=0x7fde581e7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:08.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.106+0000 7fde5f4d2640 1 --2- 172.21.15.44:0/2853292990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde580cd170 0x7fde581e7f30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fde4800c400 tx=0x7fde4800c8c0 comp rx=0 tx=0).stop 2024-04-22T06:32:08.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.107+0000 7fde5f4d2640 1 -- 172.21.15.44:0/2853292990 shutdown_connections 2024-04-22T06:32:08.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.107+0000 7fde5f4d2640 1 --2- 172.21.15.44:0/2853292990 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fde3c06c8f0 0x7fde3c06edb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:08.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.107+0000 7fde5f4d2640 1 --2- 172.21.15.44:0/2853292990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde5815e670 0x7fde58156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:08.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.107+0000 7fde5f4d2640 1 --2- 172.21.15.44:0/2853292990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde580cd170 0x7fde581e7f30 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:08.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.107+0000 7fde5f4d2640 1 --2- 172.21.15.44:0/2853292990 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde580c7d00 0x7fde581563f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:08.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.107+0000 7fde5f4d2640 1 -- 172.21.15.44:0/2853292990 >> 172.21.15.44:0/2853292990 conn(0x7fde58072fb0 msgr2=0x7fde58153030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:08.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.107+0000 7fde5f4d2640 1 -- 172.21.15.44:0/2853292990 shutdown_connections 2024-04-22T06:32:08.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.107+0000 7fde5f4d2640 1 -- 172.21.15.44:0/2853292990 wait complete. 2024-04-22T06:32:08.116 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":57,"stamp":"2024-04-22T06:32:06.558361+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600632,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":367,"num_write_kb":2482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":299,"ondisk_log_size":299,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236456,"kb_used_data":3688,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738264,"statfs":{"total":773094113280,"available":772851982336,"internally_reserved":0,"allocated":3776512,"data_stored":2153584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1406,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002522"},"pg_stats":[{"pgid":"2.1f","version":"27'13","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:55.161180+0000","last_change":"2024-04-22T06:31:06.002169+0000","last_active":"2024-04-22T06:31:55.161180+0000","last_peered":"2024-04-22T06:31:55.161180+0000","last_clean":"2024-04-22T06:31:55.161180+0000","last_became_active":"2024-04-22T06:31:06.001899+0000","last_became_peered":"2024-04-22T06:31:06.001899+0000","last_unstale":"2024-04-22T06:31:55.161180+0000","last_undegraded":"2024-04-22T06:31:55.161180+0000","last_fullsized":"2024-04-22T06:31:55.161180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:11:48.056404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.146682+0000","last_change":"2024-04-22T06:31:07.025982+0000","last_active":"2024-04-22T06:31:48.146682+0000","last_peered":"2024-04-22T06:31:48.146682+0000","last_clean":"2024-04-22T06:31:48.146682+0000","last_became_active":"2024-04-22T06:31:07.025650+0000","last_became_peered":"2024-04-22T06:31:07.025650+0000","last_unstale":"2024-04-22T06:31:48.146682+0000","last_undegraded":"2024-04-22T06:31:48.146682+0000","last_fullsized":"2024-04-22T06:31:48.146682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:58:58.029898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145570+0000","last_change":"2024-04-22T06:31:11.048083+0000","last_active":"2024-04-22T06:31:48.145570+0000","last_peered":"2024-04-22T06:31:48.145570+0000","last_clean":"2024-04-22T06:31:48.145570+0000","last_became_active":"2024-04-22T06:31:11.047682+0000","last_became_peered":"2024-04-22T06:31:11.047682+0000","last_unstale":"2024-04-22T06:31:48.145570+0000","last_undegraded":"2024-04-22T06:31:48.145570+0000","last_fullsized":"2024-04-22T06:31:48.145570+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:34:30.158935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"25'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.146793+0000","last_change":"2024-04-22T06:31:06.007484+0000","last_active":"2024-04-22T06:31:48.146793+0000","last_peered":"2024-04-22T06:31:48.146793+0000","last_clean":"2024-04-22T06:31:48.146793+0000","last_became_active":"2024-04-22T06:31:06.007204+0000","last_became_peered":"2024-04-22T06:31:06.007204+0000","last_unstale":"2024-04-22T06:31:48.146793+0000","last_undegraded":"2024-04-22T06:31:48.146793+0000","last_fullsized":"2024-04-22T06:31:48.146793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:39:18.905813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142625+0000","last_change":"2024-04-22T06:31:07.014658+0000","last_active":"2024-04-22T06:31:48.142625+0000","last_peered":"2024-04-22T06:31:48.142625+0000","last_clean":"2024-04-22T06:31:48.142625+0000","last_became_active":"2024-04-22T06:31:07.014283+0000","last_became_peered":"2024-04-22T06:31:07.014283+0000","last_unstale":"2024-04-22T06:31:48.142625+0000","last_undegraded":"2024-04-22T06:31:48.142625+0000","last_fullsized":"2024-04-22T06:31:48.142625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:43:01.159439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.190754+0000","last_change":"2024-04-22T06:31:11.045114+0000","last_active":"2024-04-22T06:31:48.190754+0000","last_peered":"2024-04-22T06:31:48.190754+0000","last_clean":"2024-04-22T06:31:48.190754+0000","last_became_active":"2024-04-22T06:31:11.044728+0000","last_became_peered":"2024-04-22T06:31:11.044728+0000","last_unstale":"2024-04-22T06:31:48.190754+0000","last_undegraded":"2024-04-22T06:31:48.190754+0000","last_fullsized":"2024-04-22T06:31:48.190754+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:48:19.215184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"27'12","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:55.161553+0000","last_change":"2024-04-22T06:31:05.995468+0000","last_active":"2024-04-22T06:31:55.161553+0000","last_peered":"2024-04-22T06:31:55.161553+0000","last_clean":"2024-04-22T06:31:55.161553+0000","last_became_active":"2024-04-22T06:31:05.995159+0000","last_became_peered":"2024-04-22T06:31:05.995159+0000","last_unstale":"2024-04-22T06:31:55.161553+0000","last_undegraded":"2024-04-22T06:31:55.161553+0000","last_fullsized":"2024-04-22T06:31:55.161553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:01:12.223718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143359+0000","last_change":"2024-04-22T06:31:07.020024+0000","last_active":"2024-04-22T06:31:48.143359+0000","last_peered":"2024-04-22T06:31:48.143359+0000","last_clean":"2024-04-22T06:31:48.143359+0000","last_became_active":"2024-04-22T06:31:07.019490+0000","last_became_peered":"2024-04-22T06:31:07.019490+0000","last_unstale":"2024-04-22T06:31:48.143359+0000","last_undegraded":"2024-04-22T06:31:48.143359+0000","last_fullsized":"2024-04-22T06:31:48.143359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:26:41.811978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.191549+0000","last_change":"2024-04-22T06:31:11.057556+0000","last_active":"2024-04-22T06:31:48.191549+0000","last_peered":"2024-04-22T06:31:48.191549+0000","last_clean":"2024-04-22T06:31:48.191549+0000","last_became_active":"2024-04-22T06:31:11.057174+0000","last_became_peered":"2024-04-22T06:31:11.057174+0000","last_unstale":"2024-04-22T06:31:48.191549+0000","last_undegraded":"2024-04-22T06:31:48.191549+0000","last_fullsized":"2024-04-22T06:31:48.191549+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:41:19.526548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"27'5","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:55.160730+0000","last_change":"2024-04-22T06:31:05.996281+0000","last_active":"2024-04-22T06:31:55.160730+0000","last_peered":"2024-04-22T06:31:55.160730+0000","last_clean":"2024-04-22T06:31:55.160730+0000","last_became_active":"2024-04-22T06:31:05.995951+0000","last_became_peered":"2024-04-22T06:31:05.995951+0000","last_unstale":"2024-04-22T06:31:55.160730+0000","last_undegraded":"2024-04-22T06:31:55.160730+0000","last_fullsized":"2024-04-22T06:31:55.160730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:01:43.732947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143274+0000","last_change":"2024-04-22T06:31:07.017755+0000","last_active":"2024-04-22T06:31:48.143274+0000","last_peered":"2024-04-22T06:31:48.143274+0000","last_clean":"2024-04-22T06:31:48.143274+0000","last_became_active":"2024-04-22T06:31:07.017420+0000","last_became_peered":"2024-04-22T06:31:07.017420+0000","last_unstale":"2024-04-22T06:31:48.143274+0000","last_undegraded":"2024-04-22T06:31:48.143274+0000","last_fullsized":"2024-04-22T06:31:48.143274+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:30:57.913020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.191016+0000","last_change":"2024-04-22T06:31:11.049497+0000","last_active":"2024-04-22T06:31:48.191016+0000","last_peered":"2024-04-22T06:31:48.191016+0000","last_clean":"2024-04-22T06:31:48.191016+0000","last_became_active":"2024-04-22T06:31:11.049079+0000","last_became_peered":"2024-04-22T06:31:11.049079+0000","last_unstale":"2024-04-22T06:31:48.191016+0000","last_undegraded":"2024-04-22T06:31:48.191016+0000","last_fullsized":"2024-04-22T06:31:48.191016+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:29:02.299691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145065+0000","last_change":"2024-04-22T06:31:07.023043+0000","last_active":"2024-04-22T06:31:48.145065+0000","last_peered":"2024-04-22T06:31:48.145065+0000","last_clean":"2024-04-22T06:31:48.145065+0000","last_became_active":"2024-04-22T06:31:07.022747+0000","last_became_peered":"2024-04-22T06:31:07.022747+0000","last_unstale":"2024-04-22T06:31:48.145065+0000","last_undegraded":"2024-04-22T06:31:48.145065+0000","last_fullsized":"2024-04-22T06:31:48.145065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:52:33.626072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"25'4","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143096+0000","last_change":"2024-04-22T06:31:05.997936+0000","last_active":"2024-04-22T06:31:48.143096+0000","last_peered":"2024-04-22T06:31:48.143096+0000","last_clean":"2024-04-22T06:31:48.143096+0000","last_became_active":"2024-04-22T06:31:05.997618+0000","last_became_peered":"2024-04-22T06:31:05.997618+0000","last_unstale":"2024-04-22T06:31:48.143096+0000","last_undegraded":"2024-04-22T06:31:48.143096+0000","last_fullsized":"2024-04-22T06:31:48.143096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:43:00.396691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.191374+0000","last_change":"2024-04-22T06:31:11.047725+0000","last_active":"2024-04-22T06:31:48.191374+0000","last_peered":"2024-04-22T06:31:48.191374+0000","last_clean":"2024-04-22T06:31:48.191374+0000","last_became_active":"2024-04-22T06:31:11.047278+0000","last_became_peered":"2024-04-22T06:31:11.047278+0000","last_unstale":"2024-04-22T06:31:48.191374+0000","last_undegraded":"2024-04-22T06:31:48.191374+0000","last_fullsized":"2024-04-22T06:31:48.191374+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:32:56.201059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145944+0000","last_change":"2024-04-22T06:31:07.021250+0000","last_active":"2024-04-22T06:31:48.145944+0000","last_peered":"2024-04-22T06:31:48.145944+0000","last_clean":"2024-04-22T06:31:48.145944+0000","last_became_active":"2024-04-22T06:31:07.020876+0000","last_became_peered":"2024-04-22T06:31:07.020876+0000","last_unstale":"2024-04-22T06:31:48.145944+0000","last_undegraded":"2024-04-22T06:31:48.145944+0000","last_fullsized":"2024-04-22T06:31:48.145944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:21:31.210819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"27'3","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:49.142316+0000","last_change":"2024-04-22T06:31:06.012775+0000","last_active":"2024-04-22T06:31:49.142316+0000","last_peered":"2024-04-22T06:31:49.142316+0000","last_clean":"2024-04-22T06:31:49.142316+0000","last_became_active":"2024-04-22T06:31:06.012474+0000","last_became_peered":"2024-04-22T06:31:06.012474+0000","last_unstale":"2024-04-22T06:31:49.142316+0000","last_undegraded":"2024-04-22T06:31:49.142316+0000","last_fullsized":"2024-04-22T06:31:49.142316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:09:48.197011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.191217+0000","last_change":"2024-04-22T06:31:11.057641+0000","last_active":"2024-04-22T06:31:48.191217+0000","last_peered":"2024-04-22T06:31:48.191217+0000","last_clean":"2024-04-22T06:31:48.191217+0000","last_became_active":"2024-04-22T06:31:11.057221+0000","last_became_peered":"2024-04-22T06:31:11.057221+0000","last_unstale":"2024-04-22T06:31:48.191217+0000","last_undegraded":"2024-04-22T06:31:48.191217+0000","last_fullsized":"2024-04-22T06:31:48.191217+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:14:57.713113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145398+0000","last_change":"2024-04-22T06:31:07.020822+0000","last_active":"2024-04-22T06:31:48.145398+0000","last_peered":"2024-04-22T06:31:48.145398+0000","last_clean":"2024-04-22T06:31:48.145398+0000","last_became_active":"2024-04-22T06:31:07.020484+0000","last_became_peered":"2024-04-22T06:31:07.020484+0000","last_unstale":"2024-04-22T06:31:48.145398+0000","last_undegraded":"2024-04-22T06:31:48.145398+0000","last_fullsized":"2024-04-22T06:31:48.145398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:18:26.522131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"27'7","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:55.160232+0000","last_change":"2024-04-22T06:31:06.000000+0000","last_active":"2024-04-22T06:31:55.160232+0000","last_peered":"2024-04-22T06:31:55.160232+0000","last_clean":"2024-04-22T06:31:55.160232+0000","last_became_active":"2024-04-22T06:31:05.999663+0000","last_became_peered":"2024-04-22T06:31:05.999663+0000","last_unstale":"2024-04-22T06:31:55.160232+0000","last_undegraded":"2024-04-22T06:31:55.160232+0000","last_fullsized":"2024-04-22T06:31:55.160232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:42.805084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.147239+0000","last_change":"2024-04-22T06:31:11.057991+0000","last_active":"2024-04-22T06:31:48.147239+0000","last_peered":"2024-04-22T06:31:48.147239+0000","last_clean":"2024-04-22T06:31:48.147239+0000","last_became_active":"2024-04-22T06:31:11.057595+0000","last_became_peered":"2024-04-22T06:31:11.057595+0000","last_unstale":"2024-04-22T06:31:48.147239+0000","last_undegraded":"2024-04-22T06:31:48.147239+0000","last_fullsized":"2024-04-22T06:31:48.147239+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:17:06.366663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.191026+0000","last_change":"2024-04-22T06:31:07.017303+0000","last_active":"2024-04-22T06:31:48.191026+0000","last_peered":"2024-04-22T06:31:48.191026+0000","last_clean":"2024-04-22T06:31:48.191026+0000","last_became_active":"2024-04-22T06:31:07.016990+0000","last_became_peered":"2024-04-22T06:31:07.016990+0000","last_unstale":"2024-04-22T06:31:48.191026+0000","last_undegraded":"2024-04-22T06:31:48.191026+0000","last_fullsized":"2024-04-22T06:31:48.191026+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:51:38.275472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"25'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143212+0000","last_change":"2024-04-22T06:31:05.997363+0000","last_active":"2024-04-22T06:31:48.143212+0000","last_peered":"2024-04-22T06:31:48.143212+0000","last_clean":"2024-04-22T06:31:48.143212+0000","last_became_active":"2024-04-22T06:31:05.996982+0000","last_became_peered":"2024-04-22T06:31:05.996982+0000","last_unstale":"2024-04-22T06:31:48.143212+0000","last_undegraded":"2024-04-22T06:31:48.143212+0000","last_fullsized":"2024-04-22T06:31:48.143212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:23:50.052130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.191397+0000","last_change":"2024-04-22T06:31:11.048507+0000","last_active":"2024-04-22T06:31:48.191397+0000","last_peered":"2024-04-22T06:31:48.191397+0000","last_clean":"2024-04-22T06:31:48.191397+0000","last_became_active":"2024-04-22T06:31:11.047722+0000","last_became_peered":"2024-04-22T06:31:11.047722+0000","last_unstale":"2024-04-22T06:31:48.191397+0000","last_undegraded":"2024-04-22T06:31:48.191397+0000","last_fullsized":"2024-04-22T06:31:48.191397+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:04:48.732029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143768+0000","last_change":"2024-04-22T06:31:07.021095+0000","last_active":"2024-04-22T06:31:48.143768+0000","last_peered":"2024-04-22T06:31:48.143768+0000","last_clean":"2024-04-22T06:31:48.143768+0000","last_became_active":"2024-04-22T06:31:07.020814+0000","last_became_peered":"2024-04-22T06:31:07.020814+0000","last_unstale":"2024-04-22T06:31:48.143768+0000","last_undegraded":"2024-04-22T06:31:48.143768+0000","last_fullsized":"2024-04-22T06:31:48.143768+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:27:08.869392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"27'7","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:55.160312+0000","last_change":"2024-04-22T06:31:05.997508+0000","last_active":"2024-04-22T06:31:55.160312+0000","last_peered":"2024-04-22T06:31:55.160312+0000","last_clean":"2024-04-22T06:31:55.160312+0000","last_became_active":"2024-04-22T06:31:05.997233+0000","last_became_peered":"2024-04-22T06:31:05.997233+0000","last_unstale":"2024-04-22T06:31:55.160312+0000","last_undegraded":"2024-04-22T06:31:55.160312+0000","last_fullsized":"2024-04-22T06:31:55.160312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:40:56.909549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.190833+0000","last_change":"2024-04-22T06:31:11.045424+0000","last_active":"2024-04-22T06:31:48.190833+0000","last_peered":"2024-04-22T06:31:48.190833+0000","last_clean":"2024-04-22T06:31:48.190833+0000","last_became_active":"2024-04-22T06:31:11.045061+0000","last_became_peered":"2024-04-22T06:31:11.045061+0000","last_unstale":"2024-04-22T06:31:48.190833+0000","last_undegraded":"2024-04-22T06:31:48.190833+0000","last_fullsized":"2024-04-22T06:31:48.190833+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:00:33.958921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.149855+0000","last_change":"2024-04-22T06:31:48.148715+0000","last_active":"2024-04-22T06:31:48.149855+0000","last_peered":"2024-04-22T06:31:48.149855+0000","last_clean":"2024-04-22T06:31:48.149855+0000","last_became_active":"2024-04-22T06:31:48.148435+0000","last_became_peered":"2024-04-22T06:31:48.148435+0000","last_unstale":"2024-04-22T06:31:48.149855+0000","last_undegraded":"2024-04-22T06:31:48.149855+0000","last_fullsized":"2024-04-22T06:31:48.149855+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:08:09.509939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"27'7","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:49.142408+0000","last_change":"2024-04-22T06:31:05.997990+0000","last_active":"2024-04-22T06:31:49.142408+0000","last_peered":"2024-04-22T06:31:49.142408+0000","last_clean":"2024-04-22T06:31:49.142408+0000","last_became_active":"2024-04-22T06:31:05.997352+0000","last_became_peered":"2024-04-22T06:31:05.997352+0000","last_unstale":"2024-04-22T06:31:49.142408+0000","last_undegraded":"2024-04-22T06:31:49.142408+0000","last_fullsized":"2024-04-22T06:31:49.142408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:15:40.726807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.144948+0000","last_change":"2024-04-22T06:31:11.053487+0000","last_active":"2024-04-22T06:31:48.144948+0000","last_peered":"2024-04-22T06:31:48.144948+0000","last_clean":"2024-04-22T06:31:48.144948+0000","last_became_active":"2024-04-22T06:31:11.053078+0000","last_became_peered":"2024-04-22T06:31:11.053078+0000","last_unstale":"2024-04-22T06:31:48.144948+0000","last_undegraded":"2024-04-22T06:31:48.144948+0000","last_fullsized":"2024-04-22T06:31:48.144948+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:23:16.130369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.147628+0000","last_change":"2024-04-22T06:31:07.020420+0000","last_active":"2024-04-22T06:31:48.147628+0000","last_peered":"2024-04-22T06:31:48.147628+0000","last_clean":"2024-04-22T06:31:48.147628+0000","last_became_active":"2024-04-22T06:31:07.020110+0000","last_became_peered":"2024-04-22T06:31:07.020110+0000","last_unstale":"2024-04-22T06:31:48.147628+0000","last_undegraded":"2024-04-22T06:31:48.147628+0000","last_fullsized":"2024-04-22T06:31:48.147628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:44:23.997057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"27'4","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:49.144779+0000","last_change":"2024-04-22T06:31:05.997357+0000","last_active":"2024-04-22T06:31:49.144779+0000","last_peered":"2024-04-22T06:31:49.144779+0000","last_clean":"2024-04-22T06:31:49.144779+0000","last_became_active":"2024-04-22T06:31:05.996913+0000","last_became_peered":"2024-04-22T06:31:05.996913+0000","last_unstale":"2024-04-22T06:31:49.144779+0000","last_undegraded":"2024-04-22T06:31:49.144779+0000","last_fullsized":"2024-04-22T06:31:49.144779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:50:51.880199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142598+0000","last_change":"2024-04-22T06:31:11.059735+0000","last_active":"2024-04-22T06:31:48.142598+0000","last_peered":"2024-04-22T06:31:48.142598+0000","last_clean":"2024-04-22T06:31:48.142598+0000","last_became_active":"2024-04-22T06:31:11.059325+0000","last_became_peered":"2024-04-22T06:31:11.059325+0000","last_unstale":"2024-04-22T06:31:48.142598+0000","last_undegraded":"2024-04-22T06:31:48.142598+0000","last_fullsized":"2024-04-22T06:31:48.142598+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:59:52.881783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143372+0000","last_change":"2024-04-22T06:31:07.026576+0000","last_active":"2024-04-22T06:31:48.143372+0000","last_peered":"2024-04-22T06:31:48.143372+0000","last_clean":"2024-04-22T06:31:48.143372+0000","last_became_active":"2024-04-22T06:31:07.026057+0000","last_became_peered":"2024-04-22T06:31:07.026057+0000","last_unstale":"2024-04-22T06:31:48.143372+0000","last_undegraded":"2024-04-22T06:31:48.143372+0000","last_fullsized":"2024-04-22T06:31:48.143372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:49:32.525918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"27'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.147781+0000","last_change":"2024-04-22T06:31:06.009043+0000","last_active":"2024-04-22T06:31:48.147781+0000","last_peered":"2024-04-22T06:31:48.147781+0000","last_clean":"2024-04-22T06:31:48.147781+0000","last_became_active":"2024-04-22T06:31:06.008752+0000","last_became_peered":"2024-04-22T06:31:06.008752+0000","last_unstale":"2024-04-22T06:31:48.147781+0000","last_undegraded":"2024-04-22T06:31:48.147781+0000","last_fullsized":"2024-04-22T06:31:48.147781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:52:34.706288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.193611+0000","last_change":"2024-04-22T06:31:48.192229+0000","last_active":"2024-04-22T06:31:48.193611+0000","last_peered":"2024-04-22T06:31:48.193611+0000","last_clean":"2024-04-22T06:31:48.193611+0000","last_became_active":"2024-04-22T06:31:48.191700+0000","last_became_peered":"2024-04-22T06:31:48.191700+0000","last_unstale":"2024-04-22T06:31:48.193611+0000","last_undegraded":"2024-04-22T06:31:48.193611+0000","last_fullsized":"2024-04-22T06:31:48.193611+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:45:48.217879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4,0],"acting":[2,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.191316+0000","last_change":"2024-04-22T06:31:07.028506+0000","last_active":"2024-04-22T06:31:48.191316+0000","last_peered":"2024-04-22T06:31:48.191316+0000","last_clean":"2024-04-22T06:31:48.191316+0000","last_became_active":"2024-04-22T06:31:07.028183+0000","last_became_peered":"2024-04-22T06:31:07.028183+0000","last_unstale":"2024-04-22T06:31:48.191316+0000","last_undegraded":"2024-04-22T06:31:48.191316+0000","last_fullsized":"2024-04-22T06:31:48.191316+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:38:45.965934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"27'5","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:55.162069+0000","last_change":"2024-04-22T06:31:06.002052+0000","last_active":"2024-04-22T06:31:55.162069+0000","last_peered":"2024-04-22T06:31:55.162069+0000","last_clean":"2024-04-22T06:31:55.162069+0000","last_became_active":"2024-04-22T06:31:06.000645+0000","last_became_peered":"2024-04-22T06:31:06.000645+0000","last_unstale":"2024-04-22T06:31:55.162069+0000","last_undegraded":"2024-04-22T06:31:55.162069+0000","last_fullsized":"2024-04-22T06:31:55.162069+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:55:38.982563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.147626+0000","last_change":"2024-04-22T06:31:11.048333+0000","last_active":"2024-04-22T06:31:48.147626+0000","last_peered":"2024-04-22T06:31:48.147626+0000","last_clean":"2024-04-22T06:31:48.147626+0000","last_became_active":"2024-04-22T06:31:11.047913+0000","last_became_peered":"2024-04-22T06:31:11.047913+0000","last_unstale":"2024-04-22T06:31:48.147626+0000","last_undegraded":"2024-04-22T06:31:48.147626+0000","last_fullsized":"2024-04-22T06:31:48.147626+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:11:52.622436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.147090+0000","last_change":"2024-04-22T06:31:07.017430+0000","last_active":"2024-04-22T06:31:48.147090+0000","last_peered":"2024-04-22T06:31:48.147090+0000","last_clean":"2024-04-22T06:31:48.147090+0000","last_became_active":"2024-04-22T06:31:07.017112+0000","last_became_peered":"2024-04-22T06:31:07.017112+0000","last_unstale":"2024-04-22T06:31:48.147090+0000","last_undegraded":"2024-04-22T06:31:48.147090+0000","last_fullsized":"2024-04-22T06:31:48.147090+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:52:18.098301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"27'5","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145524+0000","last_change":"2024-04-22T06:31:06.002202+0000","last_active":"2024-04-22T06:31:48.145524+0000","last_peered":"2024-04-22T06:31:48.145524+0000","last_clean":"2024-04-22T06:31:48.145524+0000","last_became_active":"2024-04-22T06:31:06.001916+0000","last_became_peered":"2024-04-22T06:31:06.001916+0000","last_unstale":"2024-04-22T06:31:48.145524+0000","last_undegraded":"2024-04-22T06:31:48.145524+0000","last_fullsized":"2024-04-22T06:31:48.145524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:35:52.485268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.147336+0000","last_change":"2024-04-22T06:31:11.054576+0000","last_active":"2024-04-22T06:31:48.147336+0000","last_peered":"2024-04-22T06:31:48.147336+0000","last_clean":"2024-04-22T06:31:48.147336+0000","last_became_active":"2024-04-22T06:31:11.054158+0000","last_became_peered":"2024-04-22T06:31:11.054158+0000","last_unstale":"2024-04-22T06:31:48.147336+0000","last_undegraded":"2024-04-22T06:31:48.147336+0000","last_fullsized":"2024-04-22T06:31:48.147336+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:48:50.809440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142347+0000","last_change":"2024-04-22T06:31:07.023064+0000","last_active":"2024-04-22T06:31:48.142347+0000","last_peered":"2024-04-22T06:31:48.142347+0000","last_clean":"2024-04-22T06:31:48.142347+0000","last_became_active":"2024-04-22T06:31:07.022753+0000","last_became_peered":"2024-04-22T06:31:07.022753+0000","last_unstale":"2024-04-22T06:31:48.142347+0000","last_undegraded":"2024-04-22T06:31:48.142347+0000","last_fullsized":"2024-04-22T06:31:48.142347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:25:38.039689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"27'4","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143372+0000","last_change":"2024-04-22T06:31:05.998384+0000","last_active":"2024-04-22T06:31:48.143372+0000","last_peered":"2024-04-22T06:31:48.143372+0000","last_clean":"2024-04-22T06:31:48.143372+0000","last_became_active":"2024-04-22T06:31:05.997761+0000","last_became_peered":"2024-04-22T06:31:05.997761+0000","last_unstale":"2024-04-22T06:31:48.143372+0000","last_undegraded":"2024-04-22T06:31:48.143372+0000","last_fullsized":"2024-04-22T06:31:48.143372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:07:12.014505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.149828+0000","last_change":"2024-04-22T06:31:48.149828+0000","last_active":"2024-04-22T06:31:48.149828+0000","last_peered":"2024-04-22T06:31:48.149828+0000","last_clean":"2024-04-22T06:31:48.149828+0000","last_became_active":"2024-04-22T06:31:48.149336+0000","last_became_peered":"2024-04-22T06:31:48.149336+0000","last_unstale":"2024-04-22T06:31:48.149828+0000","last_undegraded":"2024-04-22T06:31:48.149828+0000","last_fullsized":"2024-04-22T06:31:48.149828+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:09:53.968698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2,7],"acting":[0,5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145924+0000","last_change":"2024-04-22T06:31:07.026139+0000","last_active":"2024-04-22T06:31:48.145924+0000","last_peered":"2024-04-22T06:31:48.145924+0000","last_clean":"2024-04-22T06:31:48.145924+0000","last_became_active":"2024-04-22T06:31:07.025826+0000","last_became_peered":"2024-04-22T06:31:07.025826+0000","last_unstale":"2024-04-22T06:31:48.145924+0000","last_undegraded":"2024-04-22T06:31:48.145924+0000","last_fullsized":"2024-04-22T06:31:48.145924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:58:48.200581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"27'6","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:49.144921+0000","last_change":"2024-04-22T06:31:06.001815+0000","last_active":"2024-04-22T06:31:49.144921+0000","last_peered":"2024-04-22T06:31:49.144921+0000","last_clean":"2024-04-22T06:31:49.144921+0000","last_became_active":"2024-04-22T06:31:06.000629+0000","last_became_peered":"2024-04-22T06:31:06.000629+0000","last_unstale":"2024-04-22T06:31:49.144921+0000","last_undegraded":"2024-04-22T06:31:49.144921+0000","last_fullsized":"2024-04-22T06:31:49.144921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:49:31.100969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":182,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143419+0000","last_change":"2024-04-22T06:30:56.214982+0000","last_active":"2024-04-22T06:31:48.143419+0000","last_peered":"2024-04-22T06:31:48.143419+0000","last_clean":"2024-04-22T06:31:48.143419+0000","last_became_active":"2024-04-22T06:30:56.214024+0000","last_became_peered":"2024-04-22T06:30:56.214024+0000","last_unstale":"2024-04-22T06:31:48.143419+0000","last_undegraded":"2024-04-22T06:31:48.143419+0000","last_fullsized":"2024-04-22T06:31:48.143419+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:30:54.947156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:30:54.947156+0000","last_clean_scrub_stamp":"2024-04-22T06:30:54.947156+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:32:37.920238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145498+0000","last_change":"2024-04-22T06:31:11.040152+0000","last_active":"2024-04-22T06:31:48.145498+0000","last_peered":"2024-04-22T06:31:48.145498+0000","last_clean":"2024-04-22T06:31:48.145498+0000","last_became_active":"2024-04-22T06:31:11.039749+0000","last_became_peered":"2024-04-22T06:31:11.039749+0000","last_unstale":"2024-04-22T06:31:48.145498+0000","last_undegraded":"2024-04-22T06:31:48.145498+0000","last_fullsized":"2024-04-22T06:31:48.145498+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:31:23.103876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142265+0000","last_change":"2024-04-22T06:31:07.011062+0000","last_active":"2024-04-22T06:31:48.142265+0000","last_peered":"2024-04-22T06:31:48.142265+0000","last_clean":"2024-04-22T06:31:48.142265+0000","last_became_active":"2024-04-22T06:31:07.010755+0000","last_became_peered":"2024-04-22T06:31:07.010755+0000","last_unstale":"2024-04-22T06:31:48.142265+0000","last_undegraded":"2024-04-22T06:31:48.142265+0000","last_fullsized":"2024-04-22T06:31:48.142265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:57:01.373036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"27'5","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:49.141800+0000","last_change":"2024-04-22T06:31:05.998083+0000","last_active":"2024-04-22T06:31:49.141800+0000","last_peered":"2024-04-22T06:31:49.141800+0000","last_clean":"2024-04-22T06:31:49.141800+0000","last_became_active":"2024-04-22T06:31:05.997777+0000","last_became_peered":"2024-04-22T06:31:05.997777+0000","last_unstale":"2024-04-22T06:31:49.141800+0000","last_undegraded":"2024-04-22T06:31:49.141800+0000","last_fullsized":"2024-04-22T06:31:49.141800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:15:13.171198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145255+0000","last_change":"2024-04-22T06:31:11.049607+0000","last_active":"2024-04-22T06:31:48.145255+0000","last_peered":"2024-04-22T06:31:48.145255+0000","last_clean":"2024-04-22T06:31:48.145255+0000","last_became_active":"2024-04-22T06:31:11.049156+0000","last_became_peered":"2024-04-22T06:31:11.049156+0000","last_unstale":"2024-04-22T06:31:48.145255+0000","last_undegraded":"2024-04-22T06:31:48.145255+0000","last_fullsized":"2024-04-22T06:31:48.145255+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:15:48.599278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143439+0000","last_change":"2024-04-22T06:31:07.026306+0000","last_active":"2024-04-22T06:31:48.143439+0000","last_peered":"2024-04-22T06:31:48.143439+0000","last_clean":"2024-04-22T06:31:48.143439+0000","last_became_active":"2024-04-22T06:31:07.025987+0000","last_became_peered":"2024-04-22T06:31:07.025987+0000","last_unstale":"2024-04-22T06:31:48.143439+0000","last_undegraded":"2024-04-22T06:31:48.143439+0000","last_fullsized":"2024-04-22T06:31:48.143439+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:50:31.803180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"27'3","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:49.142263+0000","last_change":"2024-04-22T06:31:06.003115+0000","last_active":"2024-04-22T06:31:49.142263+0000","last_peered":"2024-04-22T06:31:49.142263+0000","last_clean":"2024-04-22T06:31:49.142263+0000","last_became_active":"2024-04-22T06:31:06.001945+0000","last_became_peered":"2024-04-22T06:31:06.001945+0000","last_unstale":"2024-04-22T06:31:49.142263+0000","last_undegraded":"2024-04-22T06:31:49.142263+0000","last_fullsized":"2024-04-22T06:31:49.142263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:59:25.770504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142789+0000","last_change":"2024-04-22T06:31:11.058841+0000","last_active":"2024-04-22T06:31:48.142789+0000","last_peered":"2024-04-22T06:31:48.142789+0000","last_clean":"2024-04-22T06:31:48.142789+0000","last_became_active":"2024-04-22T06:31:11.058407+0000","last_became_peered":"2024-04-22T06:31:11.058407+0000","last_unstale":"2024-04-22T06:31:48.142789+0000","last_undegraded":"2024-04-22T06:31:48.142789+0000","last_fullsized":"2024-04-22T06:31:48.142789+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:30:20.432210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142821+0000","last_change":"2024-04-22T06:31:07.022314+0000","last_active":"2024-04-22T06:31:48.142821+0000","last_peered":"2024-04-22T06:31:48.142821+0000","last_clean":"2024-04-22T06:31:48.142821+0000","last_became_active":"2024-04-22T06:31:07.021497+0000","last_became_peered":"2024-04-22T06:31:07.021497+0000","last_unstale":"2024-04-22T06:31:48.142821+0000","last_undegraded":"2024-04-22T06:31:48.142821+0000","last_fullsized":"2024-04-22T06:31:48.142821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:16:25.960359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'5","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:55.159169+0000","last_change":"2024-04-22T06:31:06.003805+0000","last_active":"2024-04-22T06:31:55.159169+0000","last_peered":"2024-04-22T06:31:55.159169+0000","last_clean":"2024-04-22T06:31:55.159169+0000","last_became_active":"2024-04-22T06:31:06.002735+0000","last_became_peered":"2024-04-22T06:31:06.002735+0000","last_unstale":"2024-04-22T06:31:55.159169+0000","last_undegraded":"2024-04-22T06:31:55.159169+0000","last_fullsized":"2024-04-22T06:31:55.159169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:41:16.155628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.141815+0000","last_change":"2024-04-22T06:31:11.043003+0000","last_active":"2024-04-22T06:31:48.141815+0000","last_peered":"2024-04-22T06:31:48.141815+0000","last_clean":"2024-04-22T06:31:48.141815+0000","last_became_active":"2024-04-22T06:31:11.042599+0000","last_became_peered":"2024-04-22T06:31:11.042599+0000","last_unstale":"2024-04-22T06:31:48.141815+0000","last_undegraded":"2024-04-22T06:31:48.141815+0000","last_fullsized":"2024-04-22T06:31:48.141815+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:28:58.949214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142697+0000","last_change":"2024-04-22T06:31:07.021258+0000","last_active":"2024-04-22T06:31:48.142697+0000","last_peered":"2024-04-22T06:31:48.142697+0000","last_clean":"2024-04-22T06:31:48.142697+0000","last_became_active":"2024-04-22T06:31:07.020976+0000","last_became_peered":"2024-04-22T06:31:07.020976+0000","last_unstale":"2024-04-22T06:31:48.142697+0000","last_undegraded":"2024-04-22T06:31:48.142697+0000","last_fullsized":"2024-04-22T06:31:48.142697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:19:33.377626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"27'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.191649+0000","last_change":"2024-04-22T06:31:06.002751+0000","last_active":"2024-04-22T06:31:48.191649+0000","last_peered":"2024-04-22T06:31:48.191649+0000","last_clean":"2024-04-22T06:31:48.191649+0000","last_became_active":"2024-04-22T06:31:06.002489+0000","last_became_peered":"2024-04-22T06:31:06.002489+0000","last_unstale":"2024-04-22T06:31:48.191649+0000","last_undegraded":"2024-04-22T06:31:48.191649+0000","last_fullsized":"2024-04-22T06:31:48.191649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:20:05.937529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.190750+0000","last_change":"2024-04-22T06:31:11.048954+0000","last_active":"2024-04-22T06:31:48.190750+0000","last_peered":"2024-04-22T06:31:48.190750+0000","last_clean":"2024-04-22T06:31:48.190750+0000","last_became_active":"2024-04-22T06:31:11.048214+0000","last_became_peered":"2024-04-22T06:31:11.048214+0000","last_unstale":"2024-04-22T06:31:48.190750+0000","last_undegraded":"2024-04-22T06:31:48.190750+0000","last_fullsized":"2024-04-22T06:31:48.190750+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:13:21.903203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142399+0000","last_change":"2024-04-22T06:31:07.021258+0000","last_active":"2024-04-22T06:31:48.142399+0000","last_peered":"2024-04-22T06:31:48.142399+0000","last_clean":"2024-04-22T06:31:48.142399+0000","last_became_active":"2024-04-22T06:31:07.021036+0000","last_became_peered":"2024-04-22T06:31:07.021036+0000","last_unstale":"2024-04-22T06:31:48.142399+0000","last_undegraded":"2024-04-22T06:31:48.142399+0000","last_fullsized":"2024-04-22T06:31:48.142399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:51:39.990228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'17","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:49.144742+0000","last_change":"2024-04-22T06:31:05.998270+0000","last_active":"2024-04-22T06:31:49.144742+0000","last_peered":"2024-04-22T06:31:49.144742+0000","last_clean":"2024-04-22T06:31:49.144742+0000","last_became_active":"2024-04-22T06:31:05.997925+0000","last_became_peered":"2024-04-22T06:31:05.997925+0000","last_unstale":"2024-04-22T06:31:49.144742+0000","last_undegraded":"2024-04-22T06:31:49.144742+0000","last_fullsized":"2024-04-22T06:31:49.144742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:49:44.167932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143553+0000","last_change":"2024-04-22T06:31:11.051106+0000","last_active":"2024-04-22T06:31:48.143553+0000","last_peered":"2024-04-22T06:31:48.143553+0000","last_clean":"2024-04-22T06:31:48.143553+0000","last_became_active":"2024-04-22T06:31:11.050721+0000","last_became_peered":"2024-04-22T06:31:11.050721+0000","last_unstale":"2024-04-22T06:31:48.143553+0000","last_undegraded":"2024-04-22T06:31:48.143553+0000","last_fullsized":"2024-04-22T06:31:48.143553+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:58:43.521850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145068+0000","last_change":"2024-04-22T06:31:07.020281+0000","last_active":"2024-04-22T06:31:48.145068+0000","last_peered":"2024-04-22T06:31:48.145068+0000","last_clean":"2024-04-22T06:31:48.145068+0000","last_became_active":"2024-04-22T06:31:07.020012+0000","last_became_peered":"2024-04-22T06:31:07.020012+0000","last_unstale":"2024-04-22T06:31:48.145068+0000","last_undegraded":"2024-04-22T06:31:48.145068+0000","last_fullsized":"2024-04-22T06:31:48.145068+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:20:40.301631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"27'5","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:49.139852+0000","last_change":"2024-04-22T06:31:05.999918+0000","last_active":"2024-04-22T06:31:49.139852+0000","last_peered":"2024-04-22T06:31:49.139852+0000","last_clean":"2024-04-22T06:31:49.139852+0000","last_became_active":"2024-04-22T06:31:05.999594+0000","last_became_peered":"2024-04-22T06:31:05.999594+0000","last_unstale":"2024-04-22T06:31:49.139852+0000","last_undegraded":"2024-04-22T06:31:49.139852+0000","last_fullsized":"2024-04-22T06:31:49.139852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:15:21.376196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.146252+0000","last_change":"2024-04-22T06:31:11.055940+0000","last_active":"2024-04-22T06:31:48.146252+0000","last_peered":"2024-04-22T06:31:48.146252+0000","last_clean":"2024-04-22T06:31:48.146252+0000","last_became_active":"2024-04-22T06:31:11.054873+0000","last_became_peered":"2024-04-22T06:31:11.054873+0000","last_unstale":"2024-04-22T06:31:48.146252+0000","last_undegraded":"2024-04-22T06:31:48.146252+0000","last_fullsized":"2024-04-22T06:31:48.146252+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:13:51.061199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143015+0000","last_change":"2024-04-22T06:31:07.021839+0000","last_active":"2024-04-22T06:31:48.143015+0000","last_peered":"2024-04-22T06:31:48.143015+0000","last_clean":"2024-04-22T06:31:48.143015+0000","last_became_active":"2024-04-22T06:31:07.021585+0000","last_became_peered":"2024-04-22T06:31:07.021585+0000","last_unstale":"2024-04-22T06:31:48.143015+0000","last_undegraded":"2024-04-22T06:31:48.143015+0000","last_fullsized":"2024-04-22T06:31:48.143015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:55:15.275802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"27'5","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:49.141566+0000","last_change":"2024-04-22T06:31:06.004058+0000","last_active":"2024-04-22T06:31:49.141566+0000","last_peered":"2024-04-22T06:31:49.141566+0000","last_clean":"2024-04-22T06:31:49.141566+0000","last_became_active":"2024-04-22T06:31:06.003791+0000","last_became_peered":"2024-04-22T06:31:06.003791+0000","last_unstale":"2024-04-22T06:31:49.141566+0000","last_undegraded":"2024-04-22T06:31:49.141566+0000","last_fullsized":"2024-04-22T06:31:49.141566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:03:08.913113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.146322+0000","last_change":"2024-04-22T06:31:11.054245+0000","last_active":"2024-04-22T06:31:48.146322+0000","last_peered":"2024-04-22T06:31:48.146322+0000","last_clean":"2024-04-22T06:31:48.146322+0000","last_became_active":"2024-04-22T06:31:11.053551+0000","last_became_peered":"2024-04-22T06:31:11.053551+0000","last_unstale":"2024-04-22T06:31:48.146322+0000","last_undegraded":"2024-04-22T06:31:48.146322+0000","last_fullsized":"2024-04-22T06:31:48.146322+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:34:06.312700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143337+0000","last_change":"2024-04-22T06:31:07.023273+0000","last_active":"2024-04-22T06:31:48.143337+0000","last_peered":"2024-04-22T06:31:48.143337+0000","last_clean":"2024-04-22T06:31:48.143337+0000","last_became_active":"2024-04-22T06:31:07.022978+0000","last_became_peered":"2024-04-22T06:31:07.022978+0000","last_unstale":"2024-04-22T06:31:48.143337+0000","last_undegraded":"2024-04-22T06:31:48.143337+0000","last_fullsized":"2024-04-22T06:31:48.143337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:10:50.192298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"27'9","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:55.160402+0000","last_change":"2024-04-22T06:31:06.005510+0000","last_active":"2024-04-22T06:31:55.160402+0000","last_peered":"2024-04-22T06:31:55.160402+0000","last_clean":"2024-04-22T06:31:55.160402+0000","last_became_active":"2024-04-22T06:31:06.005047+0000","last_became_peered":"2024-04-22T06:31:06.005047+0000","last_unstale":"2024-04-22T06:31:55.160402+0000","last_undegraded":"2024-04-22T06:31:55.160402+0000","last_fullsized":"2024-04-22T06:31:55.160402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:25:51.227631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143150+0000","last_change":"2024-04-22T06:31:11.046209+0000","last_active":"2024-04-22T06:31:48.143150+0000","last_peered":"2024-04-22T06:31:48.143150+0000","last_clean":"2024-04-22T06:31:48.143150+0000","last_became_active":"2024-04-22T06:31:11.045339+0000","last_became_peered":"2024-04-22T06:31:11.045339+0000","last_unstale":"2024-04-22T06:31:48.143150+0000","last_undegraded":"2024-04-22T06:31:48.143150+0000","last_fullsized":"2024-04-22T06:31:48.143150+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:39:16.659010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142146+0000","last_change":"2024-04-22T06:31:07.025901+0000","last_active":"2024-04-22T06:31:48.142146+0000","last_peered":"2024-04-22T06:31:48.142146+0000","last_clean":"2024-04-22T06:31:48.142146+0000","last_became_active":"2024-04-22T06:31:07.025590+0000","last_became_peered":"2024-04-22T06:31:07.025590+0000","last_unstale":"2024-04-22T06:31:48.142146+0000","last_undegraded":"2024-04-22T06:31:48.142146+0000","last_fullsized":"2024-04-22T06:31:48.142146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:00:26.184054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'13","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:55.161498+0000","last_change":"2024-04-22T06:31:06.004609+0000","last_active":"2024-04-22T06:31:55.161498+0000","last_peered":"2024-04-22T06:31:55.161498+0000","last_clean":"2024-04-22T06:31:55.161498+0000","last_became_active":"2024-04-22T06:31:06.004319+0000","last_became_peered":"2024-04-22T06:31:06.004319+0000","last_unstale":"2024-04-22T06:31:55.161498+0000","last_undegraded":"2024-04-22T06:31:55.161498+0000","last_fullsized":"2024-04-22T06:31:55.161498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:32:47.198906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.146760+0000","last_change":"2024-04-22T06:31:11.054706+0000","last_active":"2024-04-22T06:31:48.146760+0000","last_peered":"2024-04-22T06:31:48.146760+0000","last_clean":"2024-04-22T06:31:48.146760+0000","last_became_active":"2024-04-22T06:31:11.054307+0000","last_became_peered":"2024-04-22T06:31:11.054307+0000","last_unstale":"2024-04-22T06:31:48.146760+0000","last_undegraded":"2024-04-22T06:31:48.146760+0000","last_fullsized":"2024-04-22T06:31:48.146760+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:05:02.465114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.191527+0000","last_change":"2024-04-22T06:31:07.020739+0000","last_active":"2024-04-22T06:31:48.191527+0000","last_peered":"2024-04-22T06:31:48.191527+0000","last_clean":"2024-04-22T06:31:48.191527+0000","last_became_active":"2024-04-22T06:31:07.020070+0000","last_became_peered":"2024-04-22T06:31:07.020070+0000","last_unstale":"2024-04-22T06:31:48.191527+0000","last_undegraded":"2024-04-22T06:31:48.191527+0000","last_fullsized":"2024-04-22T06:31:48.191527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:50:22.102753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"27'5","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.147920+0000","last_change":"2024-04-22T06:31:06.003733+0000","last_active":"2024-04-22T06:31:48.147920+0000","last_peered":"2024-04-22T06:31:48.147920+0000","last_clean":"2024-04-22T06:31:48.147920+0000","last_became_active":"2024-04-22T06:31:06.003465+0000","last_became_peered":"2024-04-22T06:31:06.003465+0000","last_unstale":"2024-04-22T06:31:48.147920+0000","last_undegraded":"2024-04-22T06:31:48.147920+0000","last_fullsized":"2024-04-22T06:31:48.147920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:19:36.023311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.191203+0000","last_change":"2024-04-22T06:31:11.058041+0000","last_active":"2024-04-22T06:31:48.191203+0000","last_peered":"2024-04-22T06:31:48.191203+0000","last_clean":"2024-04-22T06:31:48.191203+0000","last_became_active":"2024-04-22T06:31:11.057243+0000","last_became_peered":"2024-04-22T06:31:11.057243+0000","last_unstale":"2024-04-22T06:31:48.191203+0000","last_undegraded":"2024-04-22T06:31:48.191203+0000","last_fullsized":"2024-04-22T06:31:48.191203+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:49:06.781140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142816+0000","last_change":"2024-04-22T06:31:07.015369+0000","last_active":"2024-04-22T06:31:48.142816+0000","last_peered":"2024-04-22T06:31:48.142816+0000","last_clean":"2024-04-22T06:31:48.142816+0000","last_became_active":"2024-04-22T06:31:07.015088+0000","last_became_peered":"2024-04-22T06:31:07.015088+0000","last_unstale":"2024-04-22T06:31:48.142816+0000","last_undegraded":"2024-04-22T06:31:48.142816+0000","last_fullsized":"2024-04-22T06:31:48.142816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:10:09.449507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"27'4","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:55.161244+0000","last_change":"2024-04-22T06:31:05.999636+0000","last_active":"2024-04-22T06:31:55.161244+0000","last_peered":"2024-04-22T06:31:55.161244+0000","last_clean":"2024-04-22T06:31:55.161244+0000","last_became_active":"2024-04-22T06:31:05.999329+0000","last_became_peered":"2024-04-22T06:31:05.999329+0000","last_unstale":"2024-04-22T06:31:55.161244+0000","last_undegraded":"2024-04-22T06:31:55.161244+0000","last_fullsized":"2024-04-22T06:31:55.161244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:09:06.670827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145230+0000","last_change":"2024-04-22T06:31:11.053530+0000","last_active":"2024-04-22T06:31:48.145230+0000","last_peered":"2024-04-22T06:31:48.145230+0000","last_clean":"2024-04-22T06:31:48.145230+0000","last_became_active":"2024-04-22T06:31:11.053100+0000","last_became_peered":"2024-04-22T06:31:11.053100+0000","last_unstale":"2024-04-22T06:31:48.145230+0000","last_undegraded":"2024-04-22T06:31:48.145230+0000","last_fullsized":"2024-04-22T06:31:48.145230+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:57:16.506840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142929+0000","last_change":"2024-04-22T06:31:07.015838+0000","last_active":"2024-04-22T06:31:48.142929+0000","last_peered":"2024-04-22T06:31:48.142929+0000","last_clean":"2024-04-22T06:31:48.142929+0000","last_became_active":"2024-04-22T06:31:07.015545+0000","last_became_peered":"2024-04-22T06:31:07.015545+0000","last_unstale":"2024-04-22T06:31:48.142929+0000","last_undegraded":"2024-04-22T06:31:48.142929+0000","last_fullsized":"2024-04-22T06:31:48.142929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:43:54.251061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"27'4","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145723+0000","last_change":"2024-04-22T06:31:06.005442+0000","last_active":"2024-04-22T06:31:48.145723+0000","last_peered":"2024-04-22T06:31:48.145723+0000","last_clean":"2024-04-22T06:31:48.145723+0000","last_became_active":"2024-04-22T06:31:06.005156+0000","last_became_peered":"2024-04-22T06:31:06.005156+0000","last_unstale":"2024-04-22T06:31:48.145723+0000","last_undegraded":"2024-04-22T06:31:48.145723+0000","last_fullsized":"2024-04-22T06:31:48.145723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:52:32.138519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145672+0000","last_change":"2024-04-22T06:31:11.046885+0000","last_active":"2024-04-22T06:31:48.145672+0000","last_peered":"2024-04-22T06:31:48.145672+0000","last_clean":"2024-04-22T06:31:48.145672+0000","last_became_active":"2024-04-22T06:31:11.046444+0000","last_became_peered":"2024-04-22T06:31:11.046444+0000","last_unstale":"2024-04-22T06:31:48.145672+0000","last_undegraded":"2024-04-22T06:31:48.145672+0000","last_fullsized":"2024-04-22T06:31:48.145672+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:17:54.523416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143477+0000","last_change":"2024-04-22T06:31:11.061784+0000","last_active":"2024-04-22T06:31:48.143477+0000","last_peered":"2024-04-22T06:31:48.143477+0000","last_clean":"2024-04-22T06:31:48.143477+0000","last_became_active":"2024-04-22T06:31:11.061382+0000","last_became_peered":"2024-04-22T06:31:11.061382+0000","last_unstale":"2024-04-22T06:31:48.143477+0000","last_undegraded":"2024-04-22T06:31:48.143477+0000","last_fullsized":"2024-04-22T06:31:48.143477+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:12:53.829206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.151807+0000","last_change":"2024-04-22T06:31:48.151807+0000","last_active":"2024-04-22T06:31:48.151807+0000","last_peered":"2024-04-22T06:31:48.151807+0000","last_clean":"2024-04-22T06:31:48.151807+0000","last_became_active":"2024-04-22T06:31:48.151525+0000","last_became_peered":"2024-04-22T06:31:48.151525+0000","last_unstale":"2024-04-22T06:31:48.151807+0000","last_undegraded":"2024-04-22T06:31:48.151807+0000","last_fullsized":"2024-04-22T06:31:48.151807+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:30:34.658957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"27'3","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:49.140808+0000","last_change":"2024-04-22T06:31:06.003080+0000","last_active":"2024-04-22T06:31:49.140808+0000","last_peered":"2024-04-22T06:31:49.140808+0000","last_clean":"2024-04-22T06:31:49.140808+0000","last_became_active":"2024-04-22T06:31:06.002725+0000","last_became_peered":"2024-04-22T06:31:06.002725+0000","last_unstale":"2024-04-22T06:31:49.140808+0000","last_undegraded":"2024-04-22T06:31:49.140808+0000","last_fullsized":"2024-04-22T06:31:49.140808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:46:38.249137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145623+0000","last_change":"2024-04-22T06:31:11.047782+0000","last_active":"2024-04-22T06:31:48.145623+0000","last_peered":"2024-04-22T06:31:48.145623+0000","last_clean":"2024-04-22T06:31:48.145623+0000","last_became_active":"2024-04-22T06:31:11.047407+0000","last_became_peered":"2024-04-22T06:31:11.047407+0000","last_unstale":"2024-04-22T06:31:48.145623+0000","last_undegraded":"2024-04-22T06:31:48.145623+0000","last_fullsized":"2024-04-22T06:31:48.145623+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:48:05.416994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145318+0000","last_change":"2024-04-22T06:31:07.025799+0000","last_active":"2024-04-22T06:31:48.145318+0000","last_peered":"2024-04-22T06:31:48.145318+0000","last_clean":"2024-04-22T06:31:48.145318+0000","last_became_active":"2024-04-22T06:31:07.025420+0000","last_became_peered":"2024-04-22T06:31:07.025420+0000","last_unstale":"2024-04-22T06:31:48.145318+0000","last_undegraded":"2024-04-22T06:31:48.145318+0000","last_fullsized":"2024-04-22T06:31:48.145318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:39:40.107621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"27'3","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.146171+0000","last_change":"2024-04-22T06:31:06.005805+0000","last_active":"2024-04-22T06:31:48.146171+0000","last_peered":"2024-04-22T06:31:48.146171+0000","last_clean":"2024-04-22T06:31:48.146171+0000","last_became_active":"2024-04-22T06:31:06.005479+0000","last_became_peered":"2024-04-22T06:31:06.005479+0000","last_unstale":"2024-04-22T06:31:48.146171+0000","last_undegraded":"2024-04-22T06:31:48.146171+0000","last_fullsized":"2024-04-22T06:31:48.146171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:59:37.140612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"27'7","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:49.140833+0000","last_change":"2024-04-22T06:31:05.998354+0000","last_active":"2024-04-22T06:31:49.140833+0000","last_peered":"2024-04-22T06:31:49.140833+0000","last_clean":"2024-04-22T06:31:49.140833+0000","last_became_active":"2024-04-22T06:31:05.998071+0000","last_became_peered":"2024-04-22T06:31:05.998071+0000","last_unstale":"2024-04-22T06:31:49.140833+0000","last_undegraded":"2024-04-22T06:31:49.140833+0000","last_fullsized":"2024-04-22T06:31:49.140833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:12:13.343435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142976+0000","last_change":"2024-04-22T06:31:07.024109+0000","last_active":"2024-04-22T06:31:48.142976+0000","last_peered":"2024-04-22T06:31:48.142976+0000","last_clean":"2024-04-22T06:31:48.142976+0000","last_became_active":"2024-04-22T06:31:07.023346+0000","last_became_peered":"2024-04-22T06:31:07.023346+0000","last_unstale":"2024-04-22T06:31:48.142976+0000","last_undegraded":"2024-04-22T06:31:48.142976+0000","last_fullsized":"2024-04-22T06:31:48.142976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:02:42.662312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145564+0000","last_change":"2024-04-22T06:31:11.047132+0000","last_active":"2024-04-22T06:31:48.145564+0000","last_peered":"2024-04-22T06:31:48.145564+0000","last_clean":"2024-04-22T06:31:48.145564+0000","last_became_active":"2024-04-22T06:31:11.046671+0000","last_became_peered":"2024-04-22T06:31:11.046671+0000","last_unstale":"2024-04-22T06:31:48.145564+0000","last_undegraded":"2024-04-22T06:31:48.145564+0000","last_fullsized":"2024-04-22T06:31:48.145564+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:41:57.706735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"25'5","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.146032+0000","last_change":"2024-04-22T06:31:06.004971+0000","last_active":"2024-04-22T06:31:48.146032+0000","last_peered":"2024-04-22T06:31:48.146032+0000","last_clean":"2024-04-22T06:31:48.146032+0000","last_became_active":"2024-04-22T06:31:06.004473+0000","last_became_peered":"2024-04-22T06:31:06.004473+0000","last_unstale":"2024-04-22T06:31:48.146032+0000","last_undegraded":"2024-04-22T06:31:48.146032+0000","last_fullsized":"2024-04-22T06:31:48.146032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:47:11.735311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.191136+0000","last_change":"2024-04-22T06:31:07.018314+0000","last_active":"2024-04-22T06:31:48.191136+0000","last_peered":"2024-04-22T06:31:48.191136+0000","last_clean":"2024-04-22T06:31:48.191136+0000","last_became_active":"2024-04-22T06:31:07.018033+0000","last_became_peered":"2024-04-22T06:31:07.018033+0000","last_unstale":"2024-04-22T06:31:48.191136+0000","last_undegraded":"2024-04-22T06:31:48.191136+0000","last_fullsized":"2024-04-22T06:31:48.191136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:06:06.099494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.192586+0000","last_change":"2024-04-22T06:31:48.192586+0000","last_active":"2024-04-22T06:31:48.192586+0000","last_peered":"2024-04-22T06:31:48.192586+0000","last_clean":"2024-04-22T06:31:48.192586+0000","last_became_active":"2024-04-22T06:31:48.192127+0000","last_became_peered":"2024-04-22T06:31:48.192127+0000","last_unstale":"2024-04-22T06:31:48.192586+0000","last_undegraded":"2024-04-22T06:31:48.192586+0000","last_fullsized":"2024-04-22T06:31:48.192586+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:58:18.053308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4,2],"acting":[3,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.144952+0000","last_change":"2024-04-22T06:31:07.024780+0000","last_active":"2024-04-22T06:31:48.144952+0000","last_peered":"2024-04-22T06:31:48.144952+0000","last_clean":"2024-04-22T06:31:48.144952+0000","last_became_active":"2024-04-22T06:31:07.024501+0000","last_became_peered":"2024-04-22T06:31:07.024501+0000","last_unstale":"2024-04-22T06:31:48.144952+0000","last_undegraded":"2024-04-22T06:31:48.144952+0000","last_fullsized":"2024-04-22T06:31:48.144952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:12:45.981232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.150411+0000","last_change":"2024-04-22T06:31:48.150411+0000","last_active":"2024-04-22T06:31:48.150411+0000","last_peered":"2024-04-22T06:31:48.150411+0000","last_clean":"2024-04-22T06:31:48.150411+0000","last_became_active":"2024-04-22T06:31:48.150131+0000","last_became_peered":"2024-04-22T06:31:48.150131+0000","last_unstale":"2024-04-22T06:31:48.150411+0000","last_undegraded":"2024-04-22T06:31:48.150411+0000","last_fullsized":"2024-04-22T06:31:48.150411+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:32:05.456771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143270+0000","last_change":"2024-04-22T06:31:07.018563+0000","last_active":"2024-04-22T06:31:48.143270+0000","last_peered":"2024-04-22T06:31:48.143270+0000","last_clean":"2024-04-22T06:31:48.143270+0000","last_became_active":"2024-04-22T06:31:07.018289+0000","last_became_peered":"2024-04-22T06:31:07.018289+0000","last_unstale":"2024-04-22T06:31:48.143270+0000","last_undegraded":"2024-04-22T06:31:48.143270+0000","last_fullsized":"2024-04-22T06:31:48.143270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:30:11.892544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142888+0000","last_change":"2024-04-22T06:31:07.022901+0000","last_active":"2024-04-22T06:31:48.142888+0000","last_peered":"2024-04-22T06:31:48.142888+0000","last_clean":"2024-04-22T06:31:48.142888+0000","last_became_active":"2024-04-22T06:31:07.022420+0000","last_became_peered":"2024-04-22T06:31:07.022420+0000","last_unstale":"2024-04-22T06:31:48.142888+0000","last_undegraded":"2024-04-22T06:31:48.142888+0000","last_fullsized":"2024-04-22T06:31:48.142888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:24:18.929896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.146584+0000","last_change":"2024-04-22T06:31:07.022875+0000","last_active":"2024-04-22T06:31:48.146584+0000","last_peered":"2024-04-22T06:31:48.146584+0000","last_clean":"2024-04-22T06:31:48.146584+0000","last_became_active":"2024-04-22T06:31:07.022545+0000","last_became_peered":"2024-04-22T06:31:07.022545+0000","last_unstale":"2024-04-22T06:31:48.146584+0000","last_undegraded":"2024-04-22T06:31:48.146584+0000","last_fullsized":"2024-04-22T06:31:48.146584+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:02:30.464508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.147729+0000","last_change":"2024-04-22T06:31:07.026367+0000","last_active":"2024-04-22T06:31:48.147729+0000","last_peered":"2024-04-22T06:31:48.147729+0000","last_clean":"2024-04-22T06:31:48.147729+0000","last_became_active":"2024-04-22T06:31:07.026085+0000","last_became_peered":"2024-04-22T06:31:07.026085+0000","last_unstale":"2024-04-22T06:31:48.147729+0000","last_undegraded":"2024-04-22T06:31:48.147729+0000","last_fullsized":"2024-04-22T06:31:48.147729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:02:55.368591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145845+0000","last_change":"2024-04-22T06:31:07.022989+0000","last_active":"2024-04-22T06:31:48.145845+0000","last_peered":"2024-04-22T06:31:48.145845+0000","last_clean":"2024-04-22T06:31:48.145845+0000","last_became_active":"2024-04-22T06:31:07.022688+0000","last_became_peered":"2024-04-22T06:31:07.022688+0000","last_unstale":"2024-04-22T06:31:48.145845+0000","last_undegraded":"2024-04-22T06:31:48.145845+0000","last_fullsized":"2024-04-22T06:31:48.145845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:24:10.910731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.146336+0000","last_change":"2024-04-22T06:31:07.026315+0000","last_active":"2024-04-22T06:31:48.146336+0000","last_peered":"2024-04-22T06:31:48.146336+0000","last_clean":"2024-04-22T06:31:48.146336+0000","last_became_active":"2024-04-22T06:31:07.025951+0000","last_became_peered":"2024-04-22T06:31:07.025951+0000","last_unstale":"2024-04-22T06:31:48.146336+0000","last_undegraded":"2024-04-22T06:31:48.146336+0000","last_fullsized":"2024-04-22T06:31:48.146336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:54:12.160058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142093+0000","last_change":"2024-04-22T06:31:07.022748+0000","last_active":"2024-04-22T06:31:48.142093+0000","last_peered":"2024-04-22T06:31:48.142093+0000","last_clean":"2024-04-22T06:31:48.142093+0000","last_became_active":"2024-04-22T06:31:07.022481+0000","last_became_peered":"2024-04-22T06:31:07.022481+0000","last_unstale":"2024-04-22T06:31:48.142093+0000","last_undegraded":"2024-04-22T06:31:48.142093+0000","last_fullsized":"2024-04-22T06:31:48.142093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:56:32.631637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145948+0000","last_change":"2024-04-22T06:31:07.021939+0000","last_active":"2024-04-22T06:31:48.145948+0000","last_peered":"2024-04-22T06:31:48.145948+0000","last_clean":"2024-04-22T06:31:48.145948+0000","last_became_active":"2024-04-22T06:31:07.021638+0000","last_became_peered":"2024-04-22T06:31:07.021638+0000","last_unstale":"2024-04-22T06:31:48.145948+0000","last_undegraded":"2024-04-22T06:31:48.145948+0000","last_fullsized":"2024-04-22T06:31:48.145948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:05:27.106092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143165+0000","last_change":"2024-04-22T06:31:07.015844+0000","last_active":"2024-04-22T06:31:48.143165+0000","last_peered":"2024-04-22T06:31:48.143165+0000","last_clean":"2024-04-22T06:31:48.143165+0000","last_became_active":"2024-04-22T06:31:07.015520+0000","last_became_peered":"2024-04-22T06:31:07.015520+0000","last_unstale":"2024-04-22T06:31:48.143165+0000","last_undegraded":"2024-04-22T06:31:48.143165+0000","last_fullsized":"2024-04-22T06:31:48.143165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:24:04.676892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142694+0000","last_change":"2024-04-22T06:31:07.023697+0000","last_active":"2024-04-22T06:31:48.142694+0000","last_peered":"2024-04-22T06:31:48.142694+0000","last_clean":"2024-04-22T06:31:48.142694+0000","last_became_active":"2024-04-22T06:31:07.023283+0000","last_became_peered":"2024-04-22T06:31:07.023283+0000","last_unstale":"2024-04-22T06:31:48.142694+0000","last_undegraded":"2024-04-22T06:31:48.142694+0000","last_fullsized":"2024-04-22T06:31:48.142694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:43:01.338104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145399+0000","last_change":"2024-04-22T06:31:07.023953+0000","last_active":"2024-04-22T06:31:48.145399+0000","last_peered":"2024-04-22T06:31:48.145399+0000","last_clean":"2024-04-22T06:31:48.145399+0000","last_became_active":"2024-04-22T06:31:07.023704+0000","last_became_peered":"2024-04-22T06:31:07.023704+0000","last_unstale":"2024-04-22T06:31:48.145399+0000","last_undegraded":"2024-04-22T06:31:48.145399+0000","last_fullsized":"2024-04-22T06:31:48.145399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:59:32.219492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142410+0000","last_change":"2024-04-22T06:31:07.027418+0000","last_active":"2024-04-22T06:31:48.142410+0000","last_peered":"2024-04-22T06:31:48.142410+0000","last_clean":"2024-04-22T06:31:48.142410+0000","last_became_active":"2024-04-22T06:31:07.027113+0000","last_became_peered":"2024-04-22T06:31:07.027113+0000","last_unstale":"2024-04-22T06:31:48.142410+0000","last_undegraded":"2024-04-22T06:31:48.142410+0000","last_fullsized":"2024-04-22T06:31:48.142410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:08:42.204674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145946+0000","last_change":"2024-04-22T06:31:07.015520+0000","last_active":"2024-04-22T06:31:48.145946+0000","last_peered":"2024-04-22T06:31:48.145946+0000","last_clean":"2024-04-22T06:31:48.145946+0000","last_became_active":"2024-04-22T06:31:07.015199+0000","last_became_peered":"2024-04-22T06:31:07.015199+0000","last_unstale":"2024-04-22T06:31:48.145946+0000","last_undegraded":"2024-04-22T06:31:48.145946+0000","last_fullsized":"2024-04-22T06:31:48.145946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:48:40.688931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142835+0000","last_change":"2024-04-22T06:31:07.016457+0000","last_active":"2024-04-22T06:31:48.142835+0000","last_peered":"2024-04-22T06:31:48.142835+0000","last_clean":"2024-04-22T06:31:48.142835+0000","last_became_active":"2024-04-22T06:31:07.016082+0000","last_became_peered":"2024-04-22T06:31:07.016082+0000","last_unstale":"2024-04-22T06:31:48.142835+0000","last_undegraded":"2024-04-22T06:31:48.142835+0000","last_fullsized":"2024-04-22T06:31:48.142835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:12:28.517065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.141985+0000","last_change":"2024-04-22T06:31:07.019955+0000","last_active":"2024-04-22T06:31:48.141985+0000","last_peered":"2024-04-22T06:31:48.141985+0000","last_clean":"2024-04-22T06:31:48.141985+0000","last_became_active":"2024-04-22T06:31:07.019174+0000","last_became_peered":"2024-04-22T06:31:07.019174+0000","last_unstale":"2024-04-22T06:31:48.141985+0000","last_undegraded":"2024-04-22T06:31:48.141985+0000","last_fullsized":"2024-04-22T06:31:48.141985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:09:07.106161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.147563+0000","last_change":"2024-04-22T06:31:07.019341+0000","last_active":"2024-04-22T06:31:48.147563+0000","last_peered":"2024-04-22T06:31:48.147563+0000","last_clean":"2024-04-22T06:31:48.147563+0000","last_became_active":"2024-04-22T06:31:07.019048+0000","last_became_peered":"2024-04-22T06:31:07.019048+0000","last_unstale":"2024-04-22T06:31:48.147563+0000","last_undegraded":"2024-04-22T06:31:48.147563+0000","last_fullsized":"2024-04-22T06:31:48.147563+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:04:19.362686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143194+0000","last_change":"2024-04-22T06:31:07.019433+0000","last_active":"2024-04-22T06:31:48.143194+0000","last_peered":"2024-04-22T06:31:48.143194+0000","last_clean":"2024-04-22T06:31:48.143194+0000","last_became_active":"2024-04-22T06:31:07.019133+0000","last_became_peered":"2024-04-22T06:31:07.019133+0000","last_unstale":"2024-04-22T06:31:48.143194+0000","last_undegraded":"2024-04-22T06:31:48.143194+0000","last_fullsized":"2024-04-22T06:31:48.143194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:01:54.466486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145847+0000","last_change":"2024-04-22T06:31:07.023014+0000","last_active":"2024-04-22T06:31:48.145847+0000","last_peered":"2024-04-22T06:31:48.145847+0000","last_clean":"2024-04-22T06:31:48.145847+0000","last_became_active":"2024-04-22T06:31:07.022716+0000","last_became_peered":"2024-04-22T06:31:07.022716+0000","last_unstale":"2024-04-22T06:31:48.145847+0000","last_undegraded":"2024-04-22T06:31:48.145847+0000","last_fullsized":"2024-04-22T06:31:48.145847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:11:47.309706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.191327+0000","last_change":"2024-04-22T06:31:07.020446+0000","last_active":"2024-04-22T06:31:48.191327+0000","last_peered":"2024-04-22T06:31:48.191327+0000","last_clean":"2024-04-22T06:31:48.191327+0000","last_became_active":"2024-04-22T06:31:07.018995+0000","last_became_peered":"2024-04-22T06:31:07.018995+0000","last_unstale":"2024-04-22T06:31:48.191327+0000","last_undegraded":"2024-04-22T06:31:48.191327+0000","last_fullsized":"2024-04-22T06:31:48.191327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:56:45.031262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.149688+0000","last_change":"2024-04-22T06:31:48.149688+0000","last_active":"2024-04-22T06:31:48.149688+0000","last_peered":"2024-04-22T06:31:48.149688+0000","last_clean":"2024-04-22T06:31:48.149688+0000","last_became_active":"2024-04-22T06:31:48.149398+0000","last_became_peered":"2024-04-22T06:31:48.149398+0000","last_unstale":"2024-04-22T06:31:48.149688+0000","last_undegraded":"2024-04-22T06:31:48.149688+0000","last_fullsized":"2024-04-22T06:31:48.149688+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:55:49.846070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145780+0000","last_change":"2024-04-22T06:31:07.014209+0000","last_active":"2024-04-22T06:31:48.145780+0000","last_peered":"2024-04-22T06:31:48.145780+0000","last_clean":"2024-04-22T06:31:48.145780+0000","last_became_active":"2024-04-22T06:31:07.013946+0000","last_became_peered":"2024-04-22T06:31:07.013946+0000","last_unstale":"2024-04-22T06:31:48.145780+0000","last_undegraded":"2024-04-22T06:31:48.145780+0000","last_fullsized":"2024-04-22T06:31:48.145780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:40:31.669158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143602+0000","last_change":"2024-04-22T06:31:07.008764+0000","last_active":"2024-04-22T06:31:48.143602+0000","last_peered":"2024-04-22T06:31:48.143602+0000","last_clean":"2024-04-22T06:31:48.143602+0000","last_became_active":"2024-04-22T06:31:07.007810+0000","last_became_peered":"2024-04-22T06:31:07.007810+0000","last_unstale":"2024-04-22T06:31:48.143602+0000","last_undegraded":"2024-04-22T06:31:48.143602+0000","last_fullsized":"2024-04-22T06:31:48.143602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:49:54.894818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145136+0000","last_change":"2024-04-22T06:31:07.022141+0000","last_active":"2024-04-22T06:31:48.145136+0000","last_peered":"2024-04-22T06:31:48.145136+0000","last_clean":"2024-04-22T06:31:48.145136+0000","last_became_active":"2024-04-22T06:31:07.021845+0000","last_became_peered":"2024-04-22T06:31:07.021845+0000","last_unstale":"2024-04-22T06:31:48.145136+0000","last_undegraded":"2024-04-22T06:31:48.145136+0000","last_fullsized":"2024-04-22T06:31:48.145136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:26:28.598104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142294+0000","last_change":"2024-04-22T06:31:07.012188+0000","last_active":"2024-04-22T06:31:48.142294+0000","last_peered":"2024-04-22T06:31:48.142294+0000","last_clean":"2024-04-22T06:31:48.142294+0000","last_became_active":"2024-04-22T06:31:07.011909+0000","last_became_peered":"2024-04-22T06:31:07.011909+0000","last_unstale":"2024-04-22T06:31:48.142294+0000","last_undegraded":"2024-04-22T06:31:48.142294+0000","last_fullsized":"2024-04-22T06:31:48.142294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:25:13.089928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143079+0000","last_change":"2024-04-22T06:31:07.020033+0000","last_active":"2024-04-22T06:31:48.143079+0000","last_peered":"2024-04-22T06:31:48.143079+0000","last_clean":"2024-04-22T06:31:48.143079+0000","last_became_active":"2024-04-22T06:31:07.019689+0000","last_became_peered":"2024-04-22T06:31:07.019689+0000","last_unstale":"2024-04-22T06:31:48.143079+0000","last_undegraded":"2024-04-22T06:31:48.143079+0000","last_fullsized":"2024-04-22T06:31:48.143079+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:09:07.209203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142885+0000","last_change":"2024-04-22T06:31:07.027287+0000","last_active":"2024-04-22T06:31:48.142885+0000","last_peered":"2024-04-22T06:31:48.142885+0000","last_clean":"2024-04-22T06:31:48.142885+0000","last_became_active":"2024-04-22T06:31:07.027015+0000","last_became_peered":"2024-04-22T06:31:07.027015+0000","last_unstale":"2024-04-22T06:31:48.142885+0000","last_undegraded":"2024-04-22T06:31:48.142885+0000","last_fullsized":"2024-04-22T06:31:48.142885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:42:49.322413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143284+0000","last_change":"2024-04-22T06:31:07.026332+0000","last_active":"2024-04-22T06:31:48.143284+0000","last_peered":"2024-04-22T06:31:48.143284+0000","last_clean":"2024-04-22T06:31:48.143284+0000","last_became_active":"2024-04-22T06:31:07.026018+0000","last_became_peered":"2024-04-22T06:31:07.026018+0000","last_unstale":"2024-04-22T06:31:48.143284+0000","last_undegraded":"2024-04-22T06:31:48.143284+0000","last_fullsized":"2024-04-22T06:31:48.143284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:27:08.560056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143261+0000","last_change":"2024-04-22T06:31:07.020497+0000","last_active":"2024-04-22T06:31:48.143261+0000","last_peered":"2024-04-22T06:31:48.143261+0000","last_clean":"2024-04-22T06:31:48.143261+0000","last_became_active":"2024-04-22T06:31:07.020194+0000","last_became_peered":"2024-04-22T06:31:07.020194+0000","last_unstale":"2024-04-22T06:31:48.143261+0000","last_undegraded":"2024-04-22T06:31:48.143261+0000","last_fullsized":"2024-04-22T06:31:48.143261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:56:59.765295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.148064+0000","last_change":"2024-04-22T06:31:07.018396+0000","last_active":"2024-04-22T06:31:48.148064+0000","last_peered":"2024-04-22T06:31:48.148064+0000","last_clean":"2024-04-22T06:31:48.148064+0000","last_became_active":"2024-04-22T06:31:07.018073+0000","last_became_peered":"2024-04-22T06:31:07.018073+0000","last_unstale":"2024-04-22T06:31:48.148064+0000","last_undegraded":"2024-04-22T06:31:48.148064+0000","last_fullsized":"2024-04-22T06:31:48.148064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:37:06.501499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142711+0000","last_change":"2024-04-22T06:31:07.014439+0000","last_active":"2024-04-22T06:31:48.142711+0000","last_peered":"2024-04-22T06:31:48.142711+0000","last_clean":"2024-04-22T06:31:48.142711+0000","last_became_active":"2024-04-22T06:31:07.014124+0000","last_became_peered":"2024-04-22T06:31:07.014124+0000","last_unstale":"2024-04-22T06:31:48.142711+0000","last_undegraded":"2024-04-22T06:31:48.142711+0000","last_fullsized":"2024-04-22T06:31:48.142711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:58:46.075068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":10264,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":208,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":20528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":182,"ondisk_log_size":182,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32076,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339764,"statfs":{"total":96636764160,"available":96603918336,"internally_reserved":0,"allocated":921600,"data_stored":711508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31488,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340352,"statfs":{"total":96636764160,"available":96604520448,"internally_reserved":0,"allocated":319488,"data_stored":123672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":339968,"data_stored":122972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31484,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340356,"statfs":{"total":96636764160,"available":96604524544,"internally_reserved":0,"allocated":315392,"data_stored":120370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":323584,"data_stored":123706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":323584,"data_stored":120502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":319488,"data_stored":120572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343932,"statfs":{"total":96636764160,"available":96608186368,"internally_reserved":0,"allocated":913408,"data_stored":710282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T06:32:08.121 INFO:tasks.quiescer.ceph_manager:clean! 2024-04-22T06:32:08.121 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T06:32:08.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.259+0000 7f35e00f2640 1 Processor -- start 2024-04-22T06:32:08.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.259+0000 7f35e00f2640 1 -- start start 2024-04-22T06:32:08.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.260+0000 7f35e00f2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f35d8155240 0x7f35d815f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:08.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.260+0000 7f35e00f2640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35d80cc480 0x7f35d80cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:08.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.260+0000 7f35e00f2640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35d80cce30 con 0x7f35d80cb320 2024-04-22T06:32:08.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.260+0000 7f35e00f2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f35d815f610 con 0x7f35d8155240 2024-04-22T06:32:08.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.260+0000 7f35e00f2640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f35d815c060 con 0x7f35d80cc480 2024-04-22T06:32:08.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.260+0000 7f35dd68d640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f35d80cb320 0x7f35d80d0ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40166/0 (socket says 172.21.15.44:40166) 2024-04-22T06:32:08.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.260+0000 7f35dd68d640 1 -- 172.21.15.44:0/2355075567 learned_addr learned my addr 172.21.15.44:0/2355075567 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:32:08.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.261+0000 7f35dde8e640 1 --2- 172.21.15.44:0/2355075567 >> v2:172.21.15.70:3300/0 conn(0x7f35d8155240 0x7f35d815f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:08.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.261+0000 7f35de68f640 1 --2- 172.21.15.44:0/2355075567 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35d80cc480 0x7f35d80cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:08.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.261+0000 7f35dce8c640 1 -- 172.21.15.44:0/2355075567 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2737491237 0 0) 0x7f35d80cce30 con 0x7f35d80cb320 2024-04-22T06:32:08.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.261+0000 7f35dce8c640 1 -- 172.21.15.44:0/2355075567 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f35c4003520 con 0x7f35d80cb320 2024-04-22T06:32:08.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.261+0000 7f35dce8c640 1 -- 172.21.15.44:0/2355075567 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2554013556 0 0) 0x7f35c4003520 con 0x7f35d80cb320 2024-04-22T06:32:08.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.261+0000 7f35dce8c640 1 -- 172.21.15.44:0/2355075567 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35d80cc480 msgr2=0x7f35d80cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:08.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.261+0000 7f35dce8c640 1 --2- 172.21.15.44:0/2355075567 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35d80cc480 0x7f35d80cc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:08.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.261+0000 7f35dce8c640 1 -- 172.21.15.44:0/2355075567 >> v2:172.21.15.70:3300/0 conn(0x7f35d8155240 msgr2=0x7f35d815f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:08.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.261+0000 7f35dce8c640 1 --2- 172.21.15.44:0/2355075567 >> v2:172.21.15.70:3300/0 conn(0x7f35d8155240 0x7f35d815f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:08.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.261+0000 7f35dce8c640 1 -- 172.21.15.44:0/2355075567 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d815b200 con 0x7f35d80cb320 2024-04-22T06:32:08.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.261+0000 7f35dce8c640 1 -- 172.21.15.44:0/2355075567 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1237339257 0 0) 0x7f35c8001670 con 0x7f35d80cb320 2024-04-22T06:32:08.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.261+0000 7f35dde8e640 1 --2- 172.21.15.44:0/2355075567 >> v2:172.21.15.70:3300/0 conn(0x7f35d8155240 0x7f35d815f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:32:08.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.261+0000 7f35dce8c640 1 -- 172.21.15.44:0/2355075567 >> v1:172.21.15.44:6789/0 conn(0x7f35d80cb320 legacy=0x7f35d80d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:08.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.261+0000 7f35dce8c640 1 --2- 172.21.15.44:0/2355075567 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35c40039e0 0x7f35c4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:08.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.261+0000 7f35dce8c640 1 --2- 172.21.15.44:0/2355075567 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35c4004800 0x7f35c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:08.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.261+0000 7f35dce8c640 1 --2- 172.21.15.44:0/2355075567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35c40051c0 0x7f35c400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:08.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.261+0000 7f35dce8c640 1 -- 172.21.15.44:0/2355075567 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f35d815c060 con 0x7f35c40051c0 2024-04-22T06:32:08.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.261+0000 7f35dce8c640 1 -- 172.21.15.44:0/2355075567 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f35c400bf00 con 0x7f35c40039e0 2024-04-22T06:32:08.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.261+0000 7f35dce8c640 1 -- 172.21.15.44:0/2355075567 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f35c400c080 con 0x7f35c4004800 2024-04-22T06:32:08.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.261+0000 7f35de68f640 1 --2- 172.21.15.44:0/2355075567 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35d80cc480 0x7f35d80cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:32:08.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.261+0000 7f35de68f640 1 --2- 172.21.15.44:0/2355075567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35c40051c0 0x7f35c400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:08.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.261+0000 7f35dd68d640 1 --2- 172.21.15.44:0/2355075567 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35c4004800 0x7f35c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:08.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.261+0000 7f35dde8e640 1 --2- 172.21.15.44:0/2355075567 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35c40039e0 0x7f35c4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:08.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.261+0000 7f35de68f640 1 -- 172.21.15.44:0/2355075567 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35c4004800 msgr2=0x7f35c4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:08.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.261+0000 7f35de68f640 1 --2- 172.21.15.44:0/2355075567 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35c4004800 0x7f35c4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:08.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.261+0000 7f35de68f640 1 -- 172.21.15.44:0/2355075567 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35c40039e0 msgr2=0x7f35c4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:08.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.261+0000 7f35de68f640 1 --2- 172.21.15.44:0/2355075567 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35c40039e0 0x7f35c4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:08.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.261+0000 7f35de68f640 1 -- 172.21.15.44:0/2355075567 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35c4003520 con 0x7f35c40051c0 2024-04-22T06:32:08.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.262+0000 7f35de68f640 1 --2- 172.21.15.44:0/2355075567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35c40051c0 0x7f35c400b9c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f35d000e1f0 tx=0x7f35c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7247467981d6577 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:08.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.262+0000 7f35dce8c640 1 -- 172.21.15.44:0/2355075567 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d0009d40 con 0x7f35c40051c0 2024-04-22T06:32:08.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.262+0000 7f35dce8c640 1 -- 172.21.15.44:0/2355075567 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35d002cce0 con 0x7f35c40051c0 2024-04-22T06:32:08.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.262+0000 7f35dce8c640 1 -- 172.21.15.44:0/2355075567 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d0003bb0 con 0x7f35c40051c0 2024-04-22T06:32:08.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.263+0000 7f35e00f2640 1 -- 172.21.15.44:0/2355075567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35c40051c0 msgr2=0x7f35c400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:08.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.263+0000 7f35e00f2640 1 --2- 172.21.15.44:0/2355075567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35c40051c0 0x7f35c400b9c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f35d000e1f0 tx=0x7f35c4005830 comp rx=0 tx=0).stop 2024-04-22T06:32:08.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.263+0000 7f35de68f640 1 -- 172.21.15.44:0/2355075567 reap_dead start 2024-04-22T06:32:08.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.263+0000 7f35e00f2640 1 -- 172.21.15.44:0/2355075567 shutdown_connections 2024-04-22T06:32:08.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.263+0000 7f35e00f2640 1 -- 172.21.15.44:0/2355075567 >> 172.21.15.44:0/2355075567 conn(0x7f35d80750d0 msgr2=0x7f35d80754e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:08.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.263+0000 7f35e00f2640 1 -- 172.21.15.44:0/2355075567 shutdown_connections 2024-04-22T06:32:08.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.263+0000 7f35e00f2640 1 -- 172.21.15.44:0/2355075567 wait complete. 2024-04-22T06:32:08.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.264+0000 7f35e00f2640 1 Processor -- start 2024-04-22T06:32:08.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.264+0000 7f35e00f2640 1 -- start start 2024-04-22T06:32:08.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.265+0000 7f35e00f2640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35c40051c0 0x7f35d81597b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:08.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.265+0000 7f35e00f2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35d815ac50 0x7f35d80cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:08.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.265+0000 7f35e00f2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35d81f4460 0x7f35d814a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:08.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.265+0000 7f35e00f2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f35d815baa0 con 0x7f35d81f4460 2024-04-22T06:32:08.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.265+0000 7f35e00f2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f35d815b7c0 con 0x7f35d815ac50 2024-04-22T06:32:08.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.265+0000 7f35e00f2640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f35d815bd80 con 0x7f35c40051c0 2024-04-22T06:32:08.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.265+0000 7f35de68f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35d81f4460 0x7f35d814a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:08.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.265+0000 7f35de68f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35d81f4460 0x7f35d814a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52532/0 (socket says 172.21.15.44:52532) 2024-04-22T06:32:08.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.265+0000 7f35de68f640 1 -- 172.21.15.44:0/183556001 learned_addr learned my addr 172.21.15.44:0/183556001 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:08.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.265+0000 7f35dd68d640 1 --2- 172.21.15.44:0/183556001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35d815ac50 0x7f35d80cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:08.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.265+0000 7f35dde8e640 1 --2- 172.21.15.44:0/183556001 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35c40051c0 0x7f35d81597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:08.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.265+0000 7f35de68f640 1 -- 172.21.15.44:0/183556001 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35c40051c0 msgr2=0x7f35d81597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:08.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.265+0000 7f35de68f640 1 --2- 172.21.15.44:0/183556001 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35c40051c0 0x7f35d81597b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:08.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.265+0000 7f35de68f640 1 -- 172.21.15.44:0/183556001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35d815ac50 msgr2=0x7f35d80cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:08.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.265+0000 7f35de68f640 1 --2- 172.21.15.44:0/183556001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35d815ac50 0x7f35d80cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:08.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.265+0000 7f35de68f640 1 -- 172.21.15.44:0/183556001 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d81f1e50 con 0x7f35d81f4460 2024-04-22T06:32:08.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.265+0000 7f35dd68d640 1 --2- 172.21.15.44:0/183556001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35d815ac50 0x7f35d80cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:32:08.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.266+0000 7f35de68f640 1 --2- 172.21.15.44:0/183556001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35d81f4460 0x7f35d814a1d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f35d0036040 tx=0x7f35d0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:08.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.266+0000 7f35ce7fc640 1 -- 172.21.15.44:0/183556001 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d0009d40 con 0x7f35d81f4460 2024-04-22T06:32:08.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.266+0000 7f35ce7fc640 1 -- 172.21.15.44:0/183556001 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35d0045070 con 0x7f35d81f4460 2024-04-22T06:32:08.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.266+0000 7f35ce7fc640 1 -- 172.21.15.44:0/183556001 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d0003e30 con 0x7f35d81f4460 2024-04-22T06:32:08.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.266+0000 7f35e00f2640 1 -- 172.21.15.44:0/183556001 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35d815a060 con 0x7f35d81f4460 2024-04-22T06:32:08.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.266+0000 7f35e00f2640 1 -- 172.21.15.44:0/183556001 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35d81f1370 con 0x7f35d81f4460 2024-04-22T06:32:08.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.267+0000 7f35e00f2640 1 -- 172.21.15.44:0/183556001 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35d8153900 con 0x7f35d81f4460 2024-04-22T06:32:08.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.267+0000 7f35ce7fc640 1 -- 172.21.15.44:0/183556001 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f35d003d040 con 0x7f35d81f4460 2024-04-22T06:32:08.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.268+0000 7f35ce7fc640 1 --2- 172.21.15.44:0/183556001 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f35bc06c920 0x7f35bc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:08.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.268+0000 7f35ce7fc640 1 -- 172.21.15.44:0/183556001 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f35d00ae670 con 0x7f35d81f4460 2024-04-22T06:32:08.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.268+0000 7f35dde8e640 1 --2- 172.21.15.44:0/183556001 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f35bc06c920 0x7f35bc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:08.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.268+0000 7f35dde8e640 1 --2- 172.21.15.44:0/183556001 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f35bc06c920 0x7f35bc06ede0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f35d4009c90 tx=0x7f35d4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:08.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.272+0000 7f35ce7fc640 1 -- 172.21.15.44:0/183556001 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f35d8153900 con 0x7f35d81f4460 2024-04-22T06:32:08.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.455+0000 7f35e00f2640 1 -- 172.21.15.44:0/183556001 --> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f35d815a340 con 0x7f35bc06c920 2024-04-22T06:32:08.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.463+0000 7f35ce7fc640 1 -- 172.21.15.44:0/183556001 <== mgr.4111 v2:172.21.15.70:6832/2570466131 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332632 (secure 0 0 0) 0x7f35d815a340 con 0x7f35bc06c920 2024-04-22T06:32:08.465 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:32:08.467 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-04-22T06:32:08.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.466+0000 7f35e00f2640 1 -- 172.21.15.44:0/183556001 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f35bc06c920 msgr2=0x7f35bc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:08.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.466+0000 7f35e00f2640 1 --2- 172.21.15.44:0/183556001 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f35bc06c920 0x7f35bc06ede0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f35d4009c90 tx=0x7f35d4008040 comp rx=0 tx=0).stop 2024-04-22T06:32:08.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.466+0000 7f35e00f2640 1 -- 172.21.15.44:0/183556001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35d81f4460 msgr2=0x7f35d814a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:08.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.466+0000 7f35e00f2640 1 --2- 172.21.15.44:0/183556001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35d81f4460 0x7f35d814a1d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f35d0036040 tx=0x7f35d0009750 comp rx=0 tx=0).stop 2024-04-22T06:32:08.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.466+0000 7f35e00f2640 1 -- 172.21.15.44:0/183556001 shutdown_connections 2024-04-22T06:32:08.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.466+0000 7f35e00f2640 1 --2- 172.21.15.44:0/183556001 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f35bc06c920 0x7f35bc06ede0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:08.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.466+0000 7f35e00f2640 1 --2- 172.21.15.44:0/183556001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35d81f4460 0x7f35d814a1d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:08.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.466+0000 7f35e00f2640 1 --2- 172.21.15.44:0/183556001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35d815ac50 0x7f35d80cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:08.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.466+0000 7f35e00f2640 1 --2- 172.21.15.44:0/183556001 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35c40051c0 0x7f35d81597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:08.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.466+0000 7f35e00f2640 1 -- 172.21.15.44:0/183556001 >> 172.21.15.44:0/183556001 conn(0x7f35d80750d0 msgr2=0x7f35d81521f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:08.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.466+0000 7f35e00f2640 1 -- 172.21.15.44:0/183556001 shutdown_connections 2024-04-22T06:32:08.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.466+0000 7f35e00f2640 1 -- 172.21.15.44:0/183556001 wait complete. 2024-04-22T06:32:08.476 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":57,"stamp":"2024-04-22T06:32:06.558361+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600632,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":367,"num_write_kb":2482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":299,"ondisk_log_size":299,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236456,"kb_used_data":3688,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738264,"statfs":{"total":773094113280,"available":772851982336,"internally_reserved":0,"allocated":3776512,"data_stored":2153584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1406,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002522"},"pg_stats":[{"pgid":"2.1f","version":"27'13","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:55.161180+0000","last_change":"2024-04-22T06:31:06.002169+0000","last_active":"2024-04-22T06:31:55.161180+0000","last_peered":"2024-04-22T06:31:55.161180+0000","last_clean":"2024-04-22T06:31:55.161180+0000","last_became_active":"2024-04-22T06:31:06.001899+0000","last_became_peered":"2024-04-22T06:31:06.001899+0000","last_unstale":"2024-04-22T06:31:55.161180+0000","last_undegraded":"2024-04-22T06:31:55.161180+0000","last_fullsized":"2024-04-22T06:31:55.161180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:11:48.056404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.146682+0000","last_change":"2024-04-22T06:31:07.025982+0000","last_active":"2024-04-22T06:31:48.146682+0000","last_peered":"2024-04-22T06:31:48.146682+0000","last_clean":"2024-04-22T06:31:48.146682+0000","last_became_active":"2024-04-22T06:31:07.025650+0000","last_became_peered":"2024-04-22T06:31:07.025650+0000","last_unstale":"2024-04-22T06:31:48.146682+0000","last_undegraded":"2024-04-22T06:31:48.146682+0000","last_fullsized":"2024-04-22T06:31:48.146682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:58:58.029898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145570+0000","last_change":"2024-04-22T06:31:11.048083+0000","last_active":"2024-04-22T06:31:48.145570+0000","last_peered":"2024-04-22T06:31:48.145570+0000","last_clean":"2024-04-22T06:31:48.145570+0000","last_became_active":"2024-04-22T06:31:11.047682+0000","last_became_peered":"2024-04-22T06:31:11.047682+0000","last_unstale":"2024-04-22T06:31:48.145570+0000","last_undegraded":"2024-04-22T06:31:48.145570+0000","last_fullsized":"2024-04-22T06:31:48.145570+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:34:30.158935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"25'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.146793+0000","last_change":"2024-04-22T06:31:06.007484+0000","last_active":"2024-04-22T06:31:48.146793+0000","last_peered":"2024-04-22T06:31:48.146793+0000","last_clean":"2024-04-22T06:31:48.146793+0000","last_became_active":"2024-04-22T06:31:06.007204+0000","last_became_peered":"2024-04-22T06:31:06.007204+0000","last_unstale":"2024-04-22T06:31:48.146793+0000","last_undegraded":"2024-04-22T06:31:48.146793+0000","last_fullsized":"2024-04-22T06:31:48.146793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:39:18.905813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142625+0000","last_change":"2024-04-22T06:31:07.014658+0000","last_active":"2024-04-22T06:31:48.142625+0000","last_peered":"2024-04-22T06:31:48.142625+0000","last_clean":"2024-04-22T06:31:48.142625+0000","last_became_active":"2024-04-22T06:31:07.014283+0000","last_became_peered":"2024-04-22T06:31:07.014283+0000","last_unstale":"2024-04-22T06:31:48.142625+0000","last_undegraded":"2024-04-22T06:31:48.142625+0000","last_fullsized":"2024-04-22T06:31:48.142625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:43:01.159439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.190754+0000","last_change":"2024-04-22T06:31:11.045114+0000","last_active":"2024-04-22T06:31:48.190754+0000","last_peered":"2024-04-22T06:31:48.190754+0000","last_clean":"2024-04-22T06:31:48.190754+0000","last_became_active":"2024-04-22T06:31:11.044728+0000","last_became_peered":"2024-04-22T06:31:11.044728+0000","last_unstale":"2024-04-22T06:31:48.190754+0000","last_undegraded":"2024-04-22T06:31:48.190754+0000","last_fullsized":"2024-04-22T06:31:48.190754+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:48:19.215184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"27'12","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:55.161553+0000","last_change":"2024-04-22T06:31:05.995468+0000","last_active":"2024-04-22T06:31:55.161553+0000","last_peered":"2024-04-22T06:31:55.161553+0000","last_clean":"2024-04-22T06:31:55.161553+0000","last_became_active":"2024-04-22T06:31:05.995159+0000","last_became_peered":"2024-04-22T06:31:05.995159+0000","last_unstale":"2024-04-22T06:31:55.161553+0000","last_undegraded":"2024-04-22T06:31:55.161553+0000","last_fullsized":"2024-04-22T06:31:55.161553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:01:12.223718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143359+0000","last_change":"2024-04-22T06:31:07.020024+0000","last_active":"2024-04-22T06:31:48.143359+0000","last_peered":"2024-04-22T06:31:48.143359+0000","last_clean":"2024-04-22T06:31:48.143359+0000","last_became_active":"2024-04-22T06:31:07.019490+0000","last_became_peered":"2024-04-22T06:31:07.019490+0000","last_unstale":"2024-04-22T06:31:48.143359+0000","last_undegraded":"2024-04-22T06:31:48.143359+0000","last_fullsized":"2024-04-22T06:31:48.143359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:26:41.811978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.191549+0000","last_change":"2024-04-22T06:31:11.057556+0000","last_active":"2024-04-22T06:31:48.191549+0000","last_peered":"2024-04-22T06:31:48.191549+0000","last_clean":"2024-04-22T06:31:48.191549+0000","last_became_active":"2024-04-22T06:31:11.057174+0000","last_became_peered":"2024-04-22T06:31:11.057174+0000","last_unstale":"2024-04-22T06:31:48.191549+0000","last_undegraded":"2024-04-22T06:31:48.191549+0000","last_fullsized":"2024-04-22T06:31:48.191549+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:41:19.526548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"27'5","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:55.160730+0000","last_change":"2024-04-22T06:31:05.996281+0000","last_active":"2024-04-22T06:31:55.160730+0000","last_peered":"2024-04-22T06:31:55.160730+0000","last_clean":"2024-04-22T06:31:55.160730+0000","last_became_active":"2024-04-22T06:31:05.995951+0000","last_became_peered":"2024-04-22T06:31:05.995951+0000","last_unstale":"2024-04-22T06:31:55.160730+0000","last_undegraded":"2024-04-22T06:31:55.160730+0000","last_fullsized":"2024-04-22T06:31:55.160730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:01:43.732947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143274+0000","last_change":"2024-04-22T06:31:07.017755+0000","last_active":"2024-04-22T06:31:48.143274+0000","last_peered":"2024-04-22T06:31:48.143274+0000","last_clean":"2024-04-22T06:31:48.143274+0000","last_became_active":"2024-04-22T06:31:07.017420+0000","last_became_peered":"2024-04-22T06:31:07.017420+0000","last_unstale":"2024-04-22T06:31:48.143274+0000","last_undegraded":"2024-04-22T06:31:48.143274+0000","last_fullsized":"2024-04-22T06:31:48.143274+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:30:57.913020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.191016+0000","last_change":"2024-04-22T06:31:11.049497+0000","last_active":"2024-04-22T06:31:48.191016+0000","last_peered":"2024-04-22T06:31:48.191016+0000","last_clean":"2024-04-22T06:31:48.191016+0000","last_became_active":"2024-04-22T06:31:11.049079+0000","last_became_peered":"2024-04-22T06:31:11.049079+0000","last_unstale":"2024-04-22T06:31:48.191016+0000","last_undegraded":"2024-04-22T06:31:48.191016+0000","last_fullsized":"2024-04-22T06:31:48.191016+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:29:02.299691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145065+0000","last_change":"2024-04-22T06:31:07.023043+0000","last_active":"2024-04-22T06:31:48.145065+0000","last_peered":"2024-04-22T06:31:48.145065+0000","last_clean":"2024-04-22T06:31:48.145065+0000","last_became_active":"2024-04-22T06:31:07.022747+0000","last_became_peered":"2024-04-22T06:31:07.022747+0000","last_unstale":"2024-04-22T06:31:48.145065+0000","last_undegraded":"2024-04-22T06:31:48.145065+0000","last_fullsized":"2024-04-22T06:31:48.145065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:52:33.626072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"25'4","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143096+0000","last_change":"2024-04-22T06:31:05.997936+0000","last_active":"2024-04-22T06:31:48.143096+0000","last_peered":"2024-04-22T06:31:48.143096+0000","last_clean":"2024-04-22T06:31:48.143096+0000","last_became_active":"2024-04-22T06:31:05.997618+0000","last_became_peered":"2024-04-22T06:31:05.997618+0000","last_unstale":"2024-04-22T06:31:48.143096+0000","last_undegraded":"2024-04-22T06:31:48.143096+0000","last_fullsized":"2024-04-22T06:31:48.143096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:43:00.396691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.191374+0000","last_change":"2024-04-22T06:31:11.047725+0000","last_active":"2024-04-22T06:31:48.191374+0000","last_peered":"2024-04-22T06:31:48.191374+0000","last_clean":"2024-04-22T06:31:48.191374+0000","last_became_active":"2024-04-22T06:31:11.047278+0000","last_became_peered":"2024-04-22T06:31:11.047278+0000","last_unstale":"2024-04-22T06:31:48.191374+0000","last_undegraded":"2024-04-22T06:31:48.191374+0000","last_fullsized":"2024-04-22T06:31:48.191374+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:32:56.201059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145944+0000","last_change":"2024-04-22T06:31:07.021250+0000","last_active":"2024-04-22T06:31:48.145944+0000","last_peered":"2024-04-22T06:31:48.145944+0000","last_clean":"2024-04-22T06:31:48.145944+0000","last_became_active":"2024-04-22T06:31:07.020876+0000","last_became_peered":"2024-04-22T06:31:07.020876+0000","last_unstale":"2024-04-22T06:31:48.145944+0000","last_undegraded":"2024-04-22T06:31:48.145944+0000","last_fullsized":"2024-04-22T06:31:48.145944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:21:31.210819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"27'3","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:49.142316+0000","last_change":"2024-04-22T06:31:06.012775+0000","last_active":"2024-04-22T06:31:49.142316+0000","last_peered":"2024-04-22T06:31:49.142316+0000","last_clean":"2024-04-22T06:31:49.142316+0000","last_became_active":"2024-04-22T06:31:06.012474+0000","last_became_peered":"2024-04-22T06:31:06.012474+0000","last_unstale":"2024-04-22T06:31:49.142316+0000","last_undegraded":"2024-04-22T06:31:49.142316+0000","last_fullsized":"2024-04-22T06:31:49.142316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:09:48.197011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.191217+0000","last_change":"2024-04-22T06:31:11.057641+0000","last_active":"2024-04-22T06:31:48.191217+0000","last_peered":"2024-04-22T06:31:48.191217+0000","last_clean":"2024-04-22T06:31:48.191217+0000","last_became_active":"2024-04-22T06:31:11.057221+0000","last_became_peered":"2024-04-22T06:31:11.057221+0000","last_unstale":"2024-04-22T06:31:48.191217+0000","last_undegraded":"2024-04-22T06:31:48.191217+0000","last_fullsized":"2024-04-22T06:31:48.191217+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:14:57.713113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145398+0000","last_change":"2024-04-22T06:31:07.020822+0000","last_active":"2024-04-22T06:31:48.145398+0000","last_peered":"2024-04-22T06:31:48.145398+0000","last_clean":"2024-04-22T06:31:48.145398+0000","last_became_active":"2024-04-22T06:31:07.020484+0000","last_became_peered":"2024-04-22T06:31:07.020484+0000","last_unstale":"2024-04-22T06:31:48.145398+0000","last_undegraded":"2024-04-22T06:31:48.145398+0000","last_fullsized":"2024-04-22T06:31:48.145398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:18:26.522131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"27'7","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:55.160232+0000","last_change":"2024-04-22T06:31:06.000000+0000","last_active":"2024-04-22T06:31:55.160232+0000","last_peered":"2024-04-22T06:31:55.160232+0000","last_clean":"2024-04-22T06:31:55.160232+0000","last_became_active":"2024-04-22T06:31:05.999663+0000","last_became_peered":"2024-04-22T06:31:05.999663+0000","last_unstale":"2024-04-22T06:31:55.160232+0000","last_undegraded":"2024-04-22T06:31:55.160232+0000","last_fullsized":"2024-04-22T06:31:55.160232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:42.805084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.147239+0000","last_change":"2024-04-22T06:31:11.057991+0000","last_active":"2024-04-22T06:31:48.147239+0000","last_peered":"2024-04-22T06:31:48.147239+0000","last_clean":"2024-04-22T06:31:48.147239+0000","last_became_active":"2024-04-22T06:31:11.057595+0000","last_became_peered":"2024-04-22T06:31:11.057595+0000","last_unstale":"2024-04-22T06:31:48.147239+0000","last_undegraded":"2024-04-22T06:31:48.147239+0000","last_fullsized":"2024-04-22T06:31:48.147239+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:17:06.366663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.191026+0000","last_change":"2024-04-22T06:31:07.017303+0000","last_active":"2024-04-22T06:31:48.191026+0000","last_peered":"2024-04-22T06:31:48.191026+0000","last_clean":"2024-04-22T06:31:48.191026+0000","last_became_active":"2024-04-22T06:31:07.016990+0000","last_became_peered":"2024-04-22T06:31:07.016990+0000","last_unstale":"2024-04-22T06:31:48.191026+0000","last_undegraded":"2024-04-22T06:31:48.191026+0000","last_fullsized":"2024-04-22T06:31:48.191026+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:51:38.275472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"25'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143212+0000","last_change":"2024-04-22T06:31:05.997363+0000","last_active":"2024-04-22T06:31:48.143212+0000","last_peered":"2024-04-22T06:31:48.143212+0000","last_clean":"2024-04-22T06:31:48.143212+0000","last_became_active":"2024-04-22T06:31:05.996982+0000","last_became_peered":"2024-04-22T06:31:05.996982+0000","last_unstale":"2024-04-22T06:31:48.143212+0000","last_undegraded":"2024-04-22T06:31:48.143212+0000","last_fullsized":"2024-04-22T06:31:48.143212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:23:50.052130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.191397+0000","last_change":"2024-04-22T06:31:11.048507+0000","last_active":"2024-04-22T06:31:48.191397+0000","last_peered":"2024-04-22T06:31:48.191397+0000","last_clean":"2024-04-22T06:31:48.191397+0000","last_became_active":"2024-04-22T06:31:11.047722+0000","last_became_peered":"2024-04-22T06:31:11.047722+0000","last_unstale":"2024-04-22T06:31:48.191397+0000","last_undegraded":"2024-04-22T06:31:48.191397+0000","last_fullsized":"2024-04-22T06:31:48.191397+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:04:48.732029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143768+0000","last_change":"2024-04-22T06:31:07.021095+0000","last_active":"2024-04-22T06:31:48.143768+0000","last_peered":"2024-04-22T06:31:48.143768+0000","last_clean":"2024-04-22T06:31:48.143768+0000","last_became_active":"2024-04-22T06:31:07.020814+0000","last_became_peered":"2024-04-22T06:31:07.020814+0000","last_unstale":"2024-04-22T06:31:48.143768+0000","last_undegraded":"2024-04-22T06:31:48.143768+0000","last_fullsized":"2024-04-22T06:31:48.143768+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:27:08.869392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"27'7","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:55.160312+0000","last_change":"2024-04-22T06:31:05.997508+0000","last_active":"2024-04-22T06:31:55.160312+0000","last_peered":"2024-04-22T06:31:55.160312+0000","last_clean":"2024-04-22T06:31:55.160312+0000","last_became_active":"2024-04-22T06:31:05.997233+0000","last_became_peered":"2024-04-22T06:31:05.997233+0000","last_unstale":"2024-04-22T06:31:55.160312+0000","last_undegraded":"2024-04-22T06:31:55.160312+0000","last_fullsized":"2024-04-22T06:31:55.160312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:40:56.909549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.190833+0000","last_change":"2024-04-22T06:31:11.045424+0000","last_active":"2024-04-22T06:31:48.190833+0000","last_peered":"2024-04-22T06:31:48.190833+0000","last_clean":"2024-04-22T06:31:48.190833+0000","last_became_active":"2024-04-22T06:31:11.045061+0000","last_became_peered":"2024-04-22T06:31:11.045061+0000","last_unstale":"2024-04-22T06:31:48.190833+0000","last_undegraded":"2024-04-22T06:31:48.190833+0000","last_fullsized":"2024-04-22T06:31:48.190833+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:00:33.958921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.149855+0000","last_change":"2024-04-22T06:31:48.148715+0000","last_active":"2024-04-22T06:31:48.149855+0000","last_peered":"2024-04-22T06:31:48.149855+0000","last_clean":"2024-04-22T06:31:48.149855+0000","last_became_active":"2024-04-22T06:31:48.148435+0000","last_became_peered":"2024-04-22T06:31:48.148435+0000","last_unstale":"2024-04-22T06:31:48.149855+0000","last_undegraded":"2024-04-22T06:31:48.149855+0000","last_fullsized":"2024-04-22T06:31:48.149855+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:08:09.509939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"27'7","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:49.142408+0000","last_change":"2024-04-22T06:31:05.997990+0000","last_active":"2024-04-22T06:31:49.142408+0000","last_peered":"2024-04-22T06:31:49.142408+0000","last_clean":"2024-04-22T06:31:49.142408+0000","last_became_active":"2024-04-22T06:31:05.997352+0000","last_became_peered":"2024-04-22T06:31:05.997352+0000","last_unstale":"2024-04-22T06:31:49.142408+0000","last_undegraded":"2024-04-22T06:31:49.142408+0000","last_fullsized":"2024-04-22T06:31:49.142408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:15:40.726807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.144948+0000","last_change":"2024-04-22T06:31:11.053487+0000","last_active":"2024-04-22T06:31:48.144948+0000","last_peered":"2024-04-22T06:31:48.144948+0000","last_clean":"2024-04-22T06:31:48.144948+0000","last_became_active":"2024-04-22T06:31:11.053078+0000","last_became_peered":"2024-04-22T06:31:11.053078+0000","last_unstale":"2024-04-22T06:31:48.144948+0000","last_undegraded":"2024-04-22T06:31:48.144948+0000","last_fullsized":"2024-04-22T06:31:48.144948+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:23:16.130369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.147628+0000","last_change":"2024-04-22T06:31:07.020420+0000","last_active":"2024-04-22T06:31:48.147628+0000","last_peered":"2024-04-22T06:31:48.147628+0000","last_clean":"2024-04-22T06:31:48.147628+0000","last_became_active":"2024-04-22T06:31:07.020110+0000","last_became_peered":"2024-04-22T06:31:07.020110+0000","last_unstale":"2024-04-22T06:31:48.147628+0000","last_undegraded":"2024-04-22T06:31:48.147628+0000","last_fullsized":"2024-04-22T06:31:48.147628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:44:23.997057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"27'4","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:49.144779+0000","last_change":"2024-04-22T06:31:05.997357+0000","last_active":"2024-04-22T06:31:49.144779+0000","last_peered":"2024-04-22T06:31:49.144779+0000","last_clean":"2024-04-22T06:31:49.144779+0000","last_became_active":"2024-04-22T06:31:05.996913+0000","last_became_peered":"2024-04-22T06:31:05.996913+0000","last_unstale":"2024-04-22T06:31:49.144779+0000","last_undegraded":"2024-04-22T06:31:49.144779+0000","last_fullsized":"2024-04-22T06:31:49.144779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:50:51.880199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142598+0000","last_change":"2024-04-22T06:31:11.059735+0000","last_active":"2024-04-22T06:31:48.142598+0000","last_peered":"2024-04-22T06:31:48.142598+0000","last_clean":"2024-04-22T06:31:48.142598+0000","last_became_active":"2024-04-22T06:31:11.059325+0000","last_became_peered":"2024-04-22T06:31:11.059325+0000","last_unstale":"2024-04-22T06:31:48.142598+0000","last_undegraded":"2024-04-22T06:31:48.142598+0000","last_fullsized":"2024-04-22T06:31:48.142598+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:59:52.881783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143372+0000","last_change":"2024-04-22T06:31:07.026576+0000","last_active":"2024-04-22T06:31:48.143372+0000","last_peered":"2024-04-22T06:31:48.143372+0000","last_clean":"2024-04-22T06:31:48.143372+0000","last_became_active":"2024-04-22T06:31:07.026057+0000","last_became_peered":"2024-04-22T06:31:07.026057+0000","last_unstale":"2024-04-22T06:31:48.143372+0000","last_undegraded":"2024-04-22T06:31:48.143372+0000","last_fullsized":"2024-04-22T06:31:48.143372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:49:32.525918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"27'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.147781+0000","last_change":"2024-04-22T06:31:06.009043+0000","last_active":"2024-04-22T06:31:48.147781+0000","last_peered":"2024-04-22T06:31:48.147781+0000","last_clean":"2024-04-22T06:31:48.147781+0000","last_became_active":"2024-04-22T06:31:06.008752+0000","last_became_peered":"2024-04-22T06:31:06.008752+0000","last_unstale":"2024-04-22T06:31:48.147781+0000","last_undegraded":"2024-04-22T06:31:48.147781+0000","last_fullsized":"2024-04-22T06:31:48.147781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:52:34.706288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.193611+0000","last_change":"2024-04-22T06:31:48.192229+0000","last_active":"2024-04-22T06:31:48.193611+0000","last_peered":"2024-04-22T06:31:48.193611+0000","last_clean":"2024-04-22T06:31:48.193611+0000","last_became_active":"2024-04-22T06:31:48.191700+0000","last_became_peered":"2024-04-22T06:31:48.191700+0000","last_unstale":"2024-04-22T06:31:48.193611+0000","last_undegraded":"2024-04-22T06:31:48.193611+0000","last_fullsized":"2024-04-22T06:31:48.193611+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:45:48.217879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4,0],"acting":[2,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.191316+0000","last_change":"2024-04-22T06:31:07.028506+0000","last_active":"2024-04-22T06:31:48.191316+0000","last_peered":"2024-04-22T06:31:48.191316+0000","last_clean":"2024-04-22T06:31:48.191316+0000","last_became_active":"2024-04-22T06:31:07.028183+0000","last_became_peered":"2024-04-22T06:31:07.028183+0000","last_unstale":"2024-04-22T06:31:48.191316+0000","last_undegraded":"2024-04-22T06:31:48.191316+0000","last_fullsized":"2024-04-22T06:31:48.191316+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:38:45.965934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"27'5","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:55.162069+0000","last_change":"2024-04-22T06:31:06.002052+0000","last_active":"2024-04-22T06:31:55.162069+0000","last_peered":"2024-04-22T06:31:55.162069+0000","last_clean":"2024-04-22T06:31:55.162069+0000","last_became_active":"2024-04-22T06:31:06.000645+0000","last_became_peered":"2024-04-22T06:31:06.000645+0000","last_unstale":"2024-04-22T06:31:55.162069+0000","last_undegraded":"2024-04-22T06:31:55.162069+0000","last_fullsized":"2024-04-22T06:31:55.162069+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:55:38.982563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.147626+0000","last_change":"2024-04-22T06:31:11.048333+0000","last_active":"2024-04-22T06:31:48.147626+0000","last_peered":"2024-04-22T06:31:48.147626+0000","last_clean":"2024-04-22T06:31:48.147626+0000","last_became_active":"2024-04-22T06:31:11.047913+0000","last_became_peered":"2024-04-22T06:31:11.047913+0000","last_unstale":"2024-04-22T06:31:48.147626+0000","last_undegraded":"2024-04-22T06:31:48.147626+0000","last_fullsized":"2024-04-22T06:31:48.147626+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:11:52.622436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.147090+0000","last_change":"2024-04-22T06:31:07.017430+0000","last_active":"2024-04-22T06:31:48.147090+0000","last_peered":"2024-04-22T06:31:48.147090+0000","last_clean":"2024-04-22T06:31:48.147090+0000","last_became_active":"2024-04-22T06:31:07.017112+0000","last_became_peered":"2024-04-22T06:31:07.017112+0000","last_unstale":"2024-04-22T06:31:48.147090+0000","last_undegraded":"2024-04-22T06:31:48.147090+0000","last_fullsized":"2024-04-22T06:31:48.147090+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:52:18.098301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"27'5","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145524+0000","last_change":"2024-04-22T06:31:06.002202+0000","last_active":"2024-04-22T06:31:48.145524+0000","last_peered":"2024-04-22T06:31:48.145524+0000","last_clean":"2024-04-22T06:31:48.145524+0000","last_became_active":"2024-04-22T06:31:06.001916+0000","last_became_peered":"2024-04-22T06:31:06.001916+0000","last_unstale":"2024-04-22T06:31:48.145524+0000","last_undegraded":"2024-04-22T06:31:48.145524+0000","last_fullsized":"2024-04-22T06:31:48.145524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:35:52.485268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.147336+0000","last_change":"2024-04-22T06:31:11.054576+0000","last_active":"2024-04-22T06:31:48.147336+0000","last_peered":"2024-04-22T06:31:48.147336+0000","last_clean":"2024-04-22T06:31:48.147336+0000","last_became_active":"2024-04-22T06:31:11.054158+0000","last_became_peered":"2024-04-22T06:31:11.054158+0000","last_unstale":"2024-04-22T06:31:48.147336+0000","last_undegraded":"2024-04-22T06:31:48.147336+0000","last_fullsized":"2024-04-22T06:31:48.147336+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:48:50.809440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142347+0000","last_change":"2024-04-22T06:31:07.023064+0000","last_active":"2024-04-22T06:31:48.142347+0000","last_peered":"2024-04-22T06:31:48.142347+0000","last_clean":"2024-04-22T06:31:48.142347+0000","last_became_active":"2024-04-22T06:31:07.022753+0000","last_became_peered":"2024-04-22T06:31:07.022753+0000","last_unstale":"2024-04-22T06:31:48.142347+0000","last_undegraded":"2024-04-22T06:31:48.142347+0000","last_fullsized":"2024-04-22T06:31:48.142347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:25:38.039689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"27'4","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143372+0000","last_change":"2024-04-22T06:31:05.998384+0000","last_active":"2024-04-22T06:31:48.143372+0000","last_peered":"2024-04-22T06:31:48.143372+0000","last_clean":"2024-04-22T06:31:48.143372+0000","last_became_active":"2024-04-22T06:31:05.997761+0000","last_became_peered":"2024-04-22T06:31:05.997761+0000","last_unstale":"2024-04-22T06:31:48.143372+0000","last_undegraded":"2024-04-22T06:31:48.143372+0000","last_fullsized":"2024-04-22T06:31:48.143372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:07:12.014505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.149828+0000","last_change":"2024-04-22T06:31:48.149828+0000","last_active":"2024-04-22T06:31:48.149828+0000","last_peered":"2024-04-22T06:31:48.149828+0000","last_clean":"2024-04-22T06:31:48.149828+0000","last_became_active":"2024-04-22T06:31:48.149336+0000","last_became_peered":"2024-04-22T06:31:48.149336+0000","last_unstale":"2024-04-22T06:31:48.149828+0000","last_undegraded":"2024-04-22T06:31:48.149828+0000","last_fullsized":"2024-04-22T06:31:48.149828+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:09:53.968698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2,7],"acting":[0,5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145924+0000","last_change":"2024-04-22T06:31:07.026139+0000","last_active":"2024-04-22T06:31:48.145924+0000","last_peered":"2024-04-22T06:31:48.145924+0000","last_clean":"2024-04-22T06:31:48.145924+0000","last_became_active":"2024-04-22T06:31:07.025826+0000","last_became_peered":"2024-04-22T06:31:07.025826+0000","last_unstale":"2024-04-22T06:31:48.145924+0000","last_undegraded":"2024-04-22T06:31:48.145924+0000","last_fullsized":"2024-04-22T06:31:48.145924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:58:48.200581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"27'6","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:49.144921+0000","last_change":"2024-04-22T06:31:06.001815+0000","last_active":"2024-04-22T06:31:49.144921+0000","last_peered":"2024-04-22T06:31:49.144921+0000","last_clean":"2024-04-22T06:31:49.144921+0000","last_became_active":"2024-04-22T06:31:06.000629+0000","last_became_peered":"2024-04-22T06:31:06.000629+0000","last_unstale":"2024-04-22T06:31:49.144921+0000","last_undegraded":"2024-04-22T06:31:49.144921+0000","last_fullsized":"2024-04-22T06:31:49.144921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:49:31.100969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":182,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143419+0000","last_change":"2024-04-22T06:30:56.214982+0000","last_active":"2024-04-22T06:31:48.143419+0000","last_peered":"2024-04-22T06:31:48.143419+0000","last_clean":"2024-04-22T06:31:48.143419+0000","last_became_active":"2024-04-22T06:30:56.214024+0000","last_became_peered":"2024-04-22T06:30:56.214024+0000","last_unstale":"2024-04-22T06:31:48.143419+0000","last_undegraded":"2024-04-22T06:31:48.143419+0000","last_fullsized":"2024-04-22T06:31:48.143419+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:30:54.947156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:30:54.947156+0000","last_clean_scrub_stamp":"2024-04-22T06:30:54.947156+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:32:37.920238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145498+0000","last_change":"2024-04-22T06:31:11.040152+0000","last_active":"2024-04-22T06:31:48.145498+0000","last_peered":"2024-04-22T06:31:48.145498+0000","last_clean":"2024-04-22T06:31:48.145498+0000","last_became_active":"2024-04-22T06:31:11.039749+0000","last_became_peered":"2024-04-22T06:31:11.039749+0000","last_unstale":"2024-04-22T06:31:48.145498+0000","last_undegraded":"2024-04-22T06:31:48.145498+0000","last_fullsized":"2024-04-22T06:31:48.145498+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:31:23.103876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142265+0000","last_change":"2024-04-22T06:31:07.011062+0000","last_active":"2024-04-22T06:31:48.142265+0000","last_peered":"2024-04-22T06:31:48.142265+0000","last_clean":"2024-04-22T06:31:48.142265+0000","last_became_active":"2024-04-22T06:31:07.010755+0000","last_became_peered":"2024-04-22T06:31:07.010755+0000","last_unstale":"2024-04-22T06:31:48.142265+0000","last_undegraded":"2024-04-22T06:31:48.142265+0000","last_fullsized":"2024-04-22T06:31:48.142265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:57:01.373036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"27'5","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:49.141800+0000","last_change":"2024-04-22T06:31:05.998083+0000","last_active":"2024-04-22T06:31:49.141800+0000","last_peered":"2024-04-22T06:31:49.141800+0000","last_clean":"2024-04-22T06:31:49.141800+0000","last_became_active":"2024-04-22T06:31:05.997777+0000","last_became_peered":"2024-04-22T06:31:05.997777+0000","last_unstale":"2024-04-22T06:31:49.141800+0000","last_undegraded":"2024-04-22T06:31:49.141800+0000","last_fullsized":"2024-04-22T06:31:49.141800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:15:13.171198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145255+0000","last_change":"2024-04-22T06:31:11.049607+0000","last_active":"2024-04-22T06:31:48.145255+0000","last_peered":"2024-04-22T06:31:48.145255+0000","last_clean":"2024-04-22T06:31:48.145255+0000","last_became_active":"2024-04-22T06:31:11.049156+0000","last_became_peered":"2024-04-22T06:31:11.049156+0000","last_unstale":"2024-04-22T06:31:48.145255+0000","last_undegraded":"2024-04-22T06:31:48.145255+0000","last_fullsized":"2024-04-22T06:31:48.145255+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:15:48.599278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143439+0000","last_change":"2024-04-22T06:31:07.026306+0000","last_active":"2024-04-22T06:31:48.143439+0000","last_peered":"2024-04-22T06:31:48.143439+0000","last_clean":"2024-04-22T06:31:48.143439+0000","last_became_active":"2024-04-22T06:31:07.025987+0000","last_became_peered":"2024-04-22T06:31:07.025987+0000","last_unstale":"2024-04-22T06:31:48.143439+0000","last_undegraded":"2024-04-22T06:31:48.143439+0000","last_fullsized":"2024-04-22T06:31:48.143439+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:50:31.803180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"27'3","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:49.142263+0000","last_change":"2024-04-22T06:31:06.003115+0000","last_active":"2024-04-22T06:31:49.142263+0000","last_peered":"2024-04-22T06:31:49.142263+0000","last_clean":"2024-04-22T06:31:49.142263+0000","last_became_active":"2024-04-22T06:31:06.001945+0000","last_became_peered":"2024-04-22T06:31:06.001945+0000","last_unstale":"2024-04-22T06:31:49.142263+0000","last_undegraded":"2024-04-22T06:31:49.142263+0000","last_fullsized":"2024-04-22T06:31:49.142263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:59:25.770504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142789+0000","last_change":"2024-04-22T06:31:11.058841+0000","last_active":"2024-04-22T06:31:48.142789+0000","last_peered":"2024-04-22T06:31:48.142789+0000","last_clean":"2024-04-22T06:31:48.142789+0000","last_became_active":"2024-04-22T06:31:11.058407+0000","last_became_peered":"2024-04-22T06:31:11.058407+0000","last_unstale":"2024-04-22T06:31:48.142789+0000","last_undegraded":"2024-04-22T06:31:48.142789+0000","last_fullsized":"2024-04-22T06:31:48.142789+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:30:20.432210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142821+0000","last_change":"2024-04-22T06:31:07.022314+0000","last_active":"2024-04-22T06:31:48.142821+0000","last_peered":"2024-04-22T06:31:48.142821+0000","last_clean":"2024-04-22T06:31:48.142821+0000","last_became_active":"2024-04-22T06:31:07.021497+0000","last_became_peered":"2024-04-22T06:31:07.021497+0000","last_unstale":"2024-04-22T06:31:48.142821+0000","last_undegraded":"2024-04-22T06:31:48.142821+0000","last_fullsized":"2024-04-22T06:31:48.142821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:16:25.960359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'5","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:55.159169+0000","last_change":"2024-04-22T06:31:06.003805+0000","last_active":"2024-04-22T06:31:55.159169+0000","last_peered":"2024-04-22T06:31:55.159169+0000","last_clean":"2024-04-22T06:31:55.159169+0000","last_became_active":"2024-04-22T06:31:06.002735+0000","last_became_peered":"2024-04-22T06:31:06.002735+0000","last_unstale":"2024-04-22T06:31:55.159169+0000","last_undegraded":"2024-04-22T06:31:55.159169+0000","last_fullsized":"2024-04-22T06:31:55.159169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:41:16.155628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.141815+0000","last_change":"2024-04-22T06:31:11.043003+0000","last_active":"2024-04-22T06:31:48.141815+0000","last_peered":"2024-04-22T06:31:48.141815+0000","last_clean":"2024-04-22T06:31:48.141815+0000","last_became_active":"2024-04-22T06:31:11.042599+0000","last_became_peered":"2024-04-22T06:31:11.042599+0000","last_unstale":"2024-04-22T06:31:48.141815+0000","last_undegraded":"2024-04-22T06:31:48.141815+0000","last_fullsized":"2024-04-22T06:31:48.141815+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:28:58.949214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142697+0000","last_change":"2024-04-22T06:31:07.021258+0000","last_active":"2024-04-22T06:31:48.142697+0000","last_peered":"2024-04-22T06:31:48.142697+0000","last_clean":"2024-04-22T06:31:48.142697+0000","last_became_active":"2024-04-22T06:31:07.020976+0000","last_became_peered":"2024-04-22T06:31:07.020976+0000","last_unstale":"2024-04-22T06:31:48.142697+0000","last_undegraded":"2024-04-22T06:31:48.142697+0000","last_fullsized":"2024-04-22T06:31:48.142697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:19:33.377626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"27'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.191649+0000","last_change":"2024-04-22T06:31:06.002751+0000","last_active":"2024-04-22T06:31:48.191649+0000","last_peered":"2024-04-22T06:31:48.191649+0000","last_clean":"2024-04-22T06:31:48.191649+0000","last_became_active":"2024-04-22T06:31:06.002489+0000","last_became_peered":"2024-04-22T06:31:06.002489+0000","last_unstale":"2024-04-22T06:31:48.191649+0000","last_undegraded":"2024-04-22T06:31:48.191649+0000","last_fullsized":"2024-04-22T06:31:48.191649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:20:05.937529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.190750+0000","last_change":"2024-04-22T06:31:11.048954+0000","last_active":"2024-04-22T06:31:48.190750+0000","last_peered":"2024-04-22T06:31:48.190750+0000","last_clean":"2024-04-22T06:31:48.190750+0000","last_became_active":"2024-04-22T06:31:11.048214+0000","last_became_peered":"2024-04-22T06:31:11.048214+0000","last_unstale":"2024-04-22T06:31:48.190750+0000","last_undegraded":"2024-04-22T06:31:48.190750+0000","last_fullsized":"2024-04-22T06:31:48.190750+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:13:21.903203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142399+0000","last_change":"2024-04-22T06:31:07.021258+0000","last_active":"2024-04-22T06:31:48.142399+0000","last_peered":"2024-04-22T06:31:48.142399+0000","last_clean":"2024-04-22T06:31:48.142399+0000","last_became_active":"2024-04-22T06:31:07.021036+0000","last_became_peered":"2024-04-22T06:31:07.021036+0000","last_unstale":"2024-04-22T06:31:48.142399+0000","last_undegraded":"2024-04-22T06:31:48.142399+0000","last_fullsized":"2024-04-22T06:31:48.142399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:51:39.990228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'17","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:49.144742+0000","last_change":"2024-04-22T06:31:05.998270+0000","last_active":"2024-04-22T06:31:49.144742+0000","last_peered":"2024-04-22T06:31:49.144742+0000","last_clean":"2024-04-22T06:31:49.144742+0000","last_became_active":"2024-04-22T06:31:05.997925+0000","last_became_peered":"2024-04-22T06:31:05.997925+0000","last_unstale":"2024-04-22T06:31:49.144742+0000","last_undegraded":"2024-04-22T06:31:49.144742+0000","last_fullsized":"2024-04-22T06:31:49.144742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:49:44.167932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143553+0000","last_change":"2024-04-22T06:31:11.051106+0000","last_active":"2024-04-22T06:31:48.143553+0000","last_peered":"2024-04-22T06:31:48.143553+0000","last_clean":"2024-04-22T06:31:48.143553+0000","last_became_active":"2024-04-22T06:31:11.050721+0000","last_became_peered":"2024-04-22T06:31:11.050721+0000","last_unstale":"2024-04-22T06:31:48.143553+0000","last_undegraded":"2024-04-22T06:31:48.143553+0000","last_fullsized":"2024-04-22T06:31:48.143553+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:58:43.521850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145068+0000","last_change":"2024-04-22T06:31:07.020281+0000","last_active":"2024-04-22T06:31:48.145068+0000","last_peered":"2024-04-22T06:31:48.145068+0000","last_clean":"2024-04-22T06:31:48.145068+0000","last_became_active":"2024-04-22T06:31:07.020012+0000","last_became_peered":"2024-04-22T06:31:07.020012+0000","last_unstale":"2024-04-22T06:31:48.145068+0000","last_undegraded":"2024-04-22T06:31:48.145068+0000","last_fullsized":"2024-04-22T06:31:48.145068+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:20:40.301631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"27'5","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:49.139852+0000","last_change":"2024-04-22T06:31:05.999918+0000","last_active":"2024-04-22T06:31:49.139852+0000","last_peered":"2024-04-22T06:31:49.139852+0000","last_clean":"2024-04-22T06:31:49.139852+0000","last_became_active":"2024-04-22T06:31:05.999594+0000","last_became_peered":"2024-04-22T06:31:05.999594+0000","last_unstale":"2024-04-22T06:31:49.139852+0000","last_undegraded":"2024-04-22T06:31:49.139852+0000","last_fullsized":"2024-04-22T06:31:49.139852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:15:21.376196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.146252+0000","last_change":"2024-04-22T06:31:11.055940+0000","last_active":"2024-04-22T06:31:48.146252+0000","last_peered":"2024-04-22T06:31:48.146252+0000","last_clean":"2024-04-22T06:31:48.146252+0000","last_became_active":"2024-04-22T06:31:11.054873+0000","last_became_peered":"2024-04-22T06:31:11.054873+0000","last_unstale":"2024-04-22T06:31:48.146252+0000","last_undegraded":"2024-04-22T06:31:48.146252+0000","last_fullsized":"2024-04-22T06:31:48.146252+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:13:51.061199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143015+0000","last_change":"2024-04-22T06:31:07.021839+0000","last_active":"2024-04-22T06:31:48.143015+0000","last_peered":"2024-04-22T06:31:48.143015+0000","last_clean":"2024-04-22T06:31:48.143015+0000","last_became_active":"2024-04-22T06:31:07.021585+0000","last_became_peered":"2024-04-22T06:31:07.021585+0000","last_unstale":"2024-04-22T06:31:48.143015+0000","last_undegraded":"2024-04-22T06:31:48.143015+0000","last_fullsized":"2024-04-22T06:31:48.143015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:55:15.275802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"27'5","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:49.141566+0000","last_change":"2024-04-22T06:31:06.004058+0000","last_active":"2024-04-22T06:31:49.141566+0000","last_peered":"2024-04-22T06:31:49.141566+0000","last_clean":"2024-04-22T06:31:49.141566+0000","last_became_active":"2024-04-22T06:31:06.003791+0000","last_became_peered":"2024-04-22T06:31:06.003791+0000","last_unstale":"2024-04-22T06:31:49.141566+0000","last_undegraded":"2024-04-22T06:31:49.141566+0000","last_fullsized":"2024-04-22T06:31:49.141566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:03:08.913113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.146322+0000","last_change":"2024-04-22T06:31:11.054245+0000","last_active":"2024-04-22T06:31:48.146322+0000","last_peered":"2024-04-22T06:31:48.146322+0000","last_clean":"2024-04-22T06:31:48.146322+0000","last_became_active":"2024-04-22T06:31:11.053551+0000","last_became_peered":"2024-04-22T06:31:11.053551+0000","last_unstale":"2024-04-22T06:31:48.146322+0000","last_undegraded":"2024-04-22T06:31:48.146322+0000","last_fullsized":"2024-04-22T06:31:48.146322+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:34:06.312700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143337+0000","last_change":"2024-04-22T06:31:07.023273+0000","last_active":"2024-04-22T06:31:48.143337+0000","last_peered":"2024-04-22T06:31:48.143337+0000","last_clean":"2024-04-22T06:31:48.143337+0000","last_became_active":"2024-04-22T06:31:07.022978+0000","last_became_peered":"2024-04-22T06:31:07.022978+0000","last_unstale":"2024-04-22T06:31:48.143337+0000","last_undegraded":"2024-04-22T06:31:48.143337+0000","last_fullsized":"2024-04-22T06:31:48.143337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:10:50.192298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"27'9","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:55.160402+0000","last_change":"2024-04-22T06:31:06.005510+0000","last_active":"2024-04-22T06:31:55.160402+0000","last_peered":"2024-04-22T06:31:55.160402+0000","last_clean":"2024-04-22T06:31:55.160402+0000","last_became_active":"2024-04-22T06:31:06.005047+0000","last_became_peered":"2024-04-22T06:31:06.005047+0000","last_unstale":"2024-04-22T06:31:55.160402+0000","last_undegraded":"2024-04-22T06:31:55.160402+0000","last_fullsized":"2024-04-22T06:31:55.160402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:25:51.227631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143150+0000","last_change":"2024-04-22T06:31:11.046209+0000","last_active":"2024-04-22T06:31:48.143150+0000","last_peered":"2024-04-22T06:31:48.143150+0000","last_clean":"2024-04-22T06:31:48.143150+0000","last_became_active":"2024-04-22T06:31:11.045339+0000","last_became_peered":"2024-04-22T06:31:11.045339+0000","last_unstale":"2024-04-22T06:31:48.143150+0000","last_undegraded":"2024-04-22T06:31:48.143150+0000","last_fullsized":"2024-04-22T06:31:48.143150+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:39:16.659010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142146+0000","last_change":"2024-04-22T06:31:07.025901+0000","last_active":"2024-04-22T06:31:48.142146+0000","last_peered":"2024-04-22T06:31:48.142146+0000","last_clean":"2024-04-22T06:31:48.142146+0000","last_became_active":"2024-04-22T06:31:07.025590+0000","last_became_peered":"2024-04-22T06:31:07.025590+0000","last_unstale":"2024-04-22T06:31:48.142146+0000","last_undegraded":"2024-04-22T06:31:48.142146+0000","last_fullsized":"2024-04-22T06:31:48.142146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:00:26.184054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'13","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:55.161498+0000","last_change":"2024-04-22T06:31:06.004609+0000","last_active":"2024-04-22T06:31:55.161498+0000","last_peered":"2024-04-22T06:31:55.161498+0000","last_clean":"2024-04-22T06:31:55.161498+0000","last_became_active":"2024-04-22T06:31:06.004319+0000","last_became_peered":"2024-04-22T06:31:06.004319+0000","last_unstale":"2024-04-22T06:31:55.161498+0000","last_undegraded":"2024-04-22T06:31:55.161498+0000","last_fullsized":"2024-04-22T06:31:55.161498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:32:47.198906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.146760+0000","last_change":"2024-04-22T06:31:11.054706+0000","last_active":"2024-04-22T06:31:48.146760+0000","last_peered":"2024-04-22T06:31:48.146760+0000","last_clean":"2024-04-22T06:31:48.146760+0000","last_became_active":"2024-04-22T06:31:11.054307+0000","last_became_peered":"2024-04-22T06:31:11.054307+0000","last_unstale":"2024-04-22T06:31:48.146760+0000","last_undegraded":"2024-04-22T06:31:48.146760+0000","last_fullsized":"2024-04-22T06:31:48.146760+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:05:02.465114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.191527+0000","last_change":"2024-04-22T06:31:07.020739+0000","last_active":"2024-04-22T06:31:48.191527+0000","last_peered":"2024-04-22T06:31:48.191527+0000","last_clean":"2024-04-22T06:31:48.191527+0000","last_became_active":"2024-04-22T06:31:07.020070+0000","last_became_peered":"2024-04-22T06:31:07.020070+0000","last_unstale":"2024-04-22T06:31:48.191527+0000","last_undegraded":"2024-04-22T06:31:48.191527+0000","last_fullsized":"2024-04-22T06:31:48.191527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:50:22.102753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"27'5","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.147920+0000","last_change":"2024-04-22T06:31:06.003733+0000","last_active":"2024-04-22T06:31:48.147920+0000","last_peered":"2024-04-22T06:31:48.147920+0000","last_clean":"2024-04-22T06:31:48.147920+0000","last_became_active":"2024-04-22T06:31:06.003465+0000","last_became_peered":"2024-04-22T06:31:06.003465+0000","last_unstale":"2024-04-22T06:31:48.147920+0000","last_undegraded":"2024-04-22T06:31:48.147920+0000","last_fullsized":"2024-04-22T06:31:48.147920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:19:36.023311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.191203+0000","last_change":"2024-04-22T06:31:11.058041+0000","last_active":"2024-04-22T06:31:48.191203+0000","last_peered":"2024-04-22T06:31:48.191203+0000","last_clean":"2024-04-22T06:31:48.191203+0000","last_became_active":"2024-04-22T06:31:11.057243+0000","last_became_peered":"2024-04-22T06:31:11.057243+0000","last_unstale":"2024-04-22T06:31:48.191203+0000","last_undegraded":"2024-04-22T06:31:48.191203+0000","last_fullsized":"2024-04-22T06:31:48.191203+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:49:06.781140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142816+0000","last_change":"2024-04-22T06:31:07.015369+0000","last_active":"2024-04-22T06:31:48.142816+0000","last_peered":"2024-04-22T06:31:48.142816+0000","last_clean":"2024-04-22T06:31:48.142816+0000","last_became_active":"2024-04-22T06:31:07.015088+0000","last_became_peered":"2024-04-22T06:31:07.015088+0000","last_unstale":"2024-04-22T06:31:48.142816+0000","last_undegraded":"2024-04-22T06:31:48.142816+0000","last_fullsized":"2024-04-22T06:31:48.142816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:10:09.449507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"27'4","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:55.161244+0000","last_change":"2024-04-22T06:31:05.999636+0000","last_active":"2024-04-22T06:31:55.161244+0000","last_peered":"2024-04-22T06:31:55.161244+0000","last_clean":"2024-04-22T06:31:55.161244+0000","last_became_active":"2024-04-22T06:31:05.999329+0000","last_became_peered":"2024-04-22T06:31:05.999329+0000","last_unstale":"2024-04-22T06:31:55.161244+0000","last_undegraded":"2024-04-22T06:31:55.161244+0000","last_fullsized":"2024-04-22T06:31:55.161244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:09:06.670827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145230+0000","last_change":"2024-04-22T06:31:11.053530+0000","last_active":"2024-04-22T06:31:48.145230+0000","last_peered":"2024-04-22T06:31:48.145230+0000","last_clean":"2024-04-22T06:31:48.145230+0000","last_became_active":"2024-04-22T06:31:11.053100+0000","last_became_peered":"2024-04-22T06:31:11.053100+0000","last_unstale":"2024-04-22T06:31:48.145230+0000","last_undegraded":"2024-04-22T06:31:48.145230+0000","last_fullsized":"2024-04-22T06:31:48.145230+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:57:16.506840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142929+0000","last_change":"2024-04-22T06:31:07.015838+0000","last_active":"2024-04-22T06:31:48.142929+0000","last_peered":"2024-04-22T06:31:48.142929+0000","last_clean":"2024-04-22T06:31:48.142929+0000","last_became_active":"2024-04-22T06:31:07.015545+0000","last_became_peered":"2024-04-22T06:31:07.015545+0000","last_unstale":"2024-04-22T06:31:48.142929+0000","last_undegraded":"2024-04-22T06:31:48.142929+0000","last_fullsized":"2024-04-22T06:31:48.142929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:43:54.251061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"27'4","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145723+0000","last_change":"2024-04-22T06:31:06.005442+0000","last_active":"2024-04-22T06:31:48.145723+0000","last_peered":"2024-04-22T06:31:48.145723+0000","last_clean":"2024-04-22T06:31:48.145723+0000","last_became_active":"2024-04-22T06:31:06.005156+0000","last_became_peered":"2024-04-22T06:31:06.005156+0000","last_unstale":"2024-04-22T06:31:48.145723+0000","last_undegraded":"2024-04-22T06:31:48.145723+0000","last_fullsized":"2024-04-22T06:31:48.145723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:52:32.138519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145672+0000","last_change":"2024-04-22T06:31:11.046885+0000","last_active":"2024-04-22T06:31:48.145672+0000","last_peered":"2024-04-22T06:31:48.145672+0000","last_clean":"2024-04-22T06:31:48.145672+0000","last_became_active":"2024-04-22T06:31:11.046444+0000","last_became_peered":"2024-04-22T06:31:11.046444+0000","last_unstale":"2024-04-22T06:31:48.145672+0000","last_undegraded":"2024-04-22T06:31:48.145672+0000","last_fullsized":"2024-04-22T06:31:48.145672+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:17:54.523416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143477+0000","last_change":"2024-04-22T06:31:11.061784+0000","last_active":"2024-04-22T06:31:48.143477+0000","last_peered":"2024-04-22T06:31:48.143477+0000","last_clean":"2024-04-22T06:31:48.143477+0000","last_became_active":"2024-04-22T06:31:11.061382+0000","last_became_peered":"2024-04-22T06:31:11.061382+0000","last_unstale":"2024-04-22T06:31:48.143477+0000","last_undegraded":"2024-04-22T06:31:48.143477+0000","last_fullsized":"2024-04-22T06:31:48.143477+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:12:53.829206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.151807+0000","last_change":"2024-04-22T06:31:48.151807+0000","last_active":"2024-04-22T06:31:48.151807+0000","last_peered":"2024-04-22T06:31:48.151807+0000","last_clean":"2024-04-22T06:31:48.151807+0000","last_became_active":"2024-04-22T06:31:48.151525+0000","last_became_peered":"2024-04-22T06:31:48.151525+0000","last_unstale":"2024-04-22T06:31:48.151807+0000","last_undegraded":"2024-04-22T06:31:48.151807+0000","last_fullsized":"2024-04-22T06:31:48.151807+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:30:34.658957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"27'3","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:49.140808+0000","last_change":"2024-04-22T06:31:06.003080+0000","last_active":"2024-04-22T06:31:49.140808+0000","last_peered":"2024-04-22T06:31:49.140808+0000","last_clean":"2024-04-22T06:31:49.140808+0000","last_became_active":"2024-04-22T06:31:06.002725+0000","last_became_peered":"2024-04-22T06:31:06.002725+0000","last_unstale":"2024-04-22T06:31:49.140808+0000","last_undegraded":"2024-04-22T06:31:49.140808+0000","last_fullsized":"2024-04-22T06:31:49.140808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:46:38.249137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145623+0000","last_change":"2024-04-22T06:31:11.047782+0000","last_active":"2024-04-22T06:31:48.145623+0000","last_peered":"2024-04-22T06:31:48.145623+0000","last_clean":"2024-04-22T06:31:48.145623+0000","last_became_active":"2024-04-22T06:31:11.047407+0000","last_became_peered":"2024-04-22T06:31:11.047407+0000","last_unstale":"2024-04-22T06:31:48.145623+0000","last_undegraded":"2024-04-22T06:31:48.145623+0000","last_fullsized":"2024-04-22T06:31:48.145623+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:48:05.416994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145318+0000","last_change":"2024-04-22T06:31:07.025799+0000","last_active":"2024-04-22T06:31:48.145318+0000","last_peered":"2024-04-22T06:31:48.145318+0000","last_clean":"2024-04-22T06:31:48.145318+0000","last_became_active":"2024-04-22T06:31:07.025420+0000","last_became_peered":"2024-04-22T06:31:07.025420+0000","last_unstale":"2024-04-22T06:31:48.145318+0000","last_undegraded":"2024-04-22T06:31:48.145318+0000","last_fullsized":"2024-04-22T06:31:48.145318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:39:40.107621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"27'3","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.146171+0000","last_change":"2024-04-22T06:31:06.005805+0000","last_active":"2024-04-22T06:31:48.146171+0000","last_peered":"2024-04-22T06:31:48.146171+0000","last_clean":"2024-04-22T06:31:48.146171+0000","last_became_active":"2024-04-22T06:31:06.005479+0000","last_became_peered":"2024-04-22T06:31:06.005479+0000","last_unstale":"2024-04-22T06:31:48.146171+0000","last_undegraded":"2024-04-22T06:31:48.146171+0000","last_fullsized":"2024-04-22T06:31:48.146171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:59:37.140612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"27'7","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:49.140833+0000","last_change":"2024-04-22T06:31:05.998354+0000","last_active":"2024-04-22T06:31:49.140833+0000","last_peered":"2024-04-22T06:31:49.140833+0000","last_clean":"2024-04-22T06:31:49.140833+0000","last_became_active":"2024-04-22T06:31:05.998071+0000","last_became_peered":"2024-04-22T06:31:05.998071+0000","last_unstale":"2024-04-22T06:31:49.140833+0000","last_undegraded":"2024-04-22T06:31:49.140833+0000","last_fullsized":"2024-04-22T06:31:49.140833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:12:13.343435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142976+0000","last_change":"2024-04-22T06:31:07.024109+0000","last_active":"2024-04-22T06:31:48.142976+0000","last_peered":"2024-04-22T06:31:48.142976+0000","last_clean":"2024-04-22T06:31:48.142976+0000","last_became_active":"2024-04-22T06:31:07.023346+0000","last_became_peered":"2024-04-22T06:31:07.023346+0000","last_unstale":"2024-04-22T06:31:48.142976+0000","last_undegraded":"2024-04-22T06:31:48.142976+0000","last_fullsized":"2024-04-22T06:31:48.142976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:02:42.662312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145564+0000","last_change":"2024-04-22T06:31:11.047132+0000","last_active":"2024-04-22T06:31:48.145564+0000","last_peered":"2024-04-22T06:31:48.145564+0000","last_clean":"2024-04-22T06:31:48.145564+0000","last_became_active":"2024-04-22T06:31:11.046671+0000","last_became_peered":"2024-04-22T06:31:11.046671+0000","last_unstale":"2024-04-22T06:31:48.145564+0000","last_undegraded":"2024-04-22T06:31:48.145564+0000","last_fullsized":"2024-04-22T06:31:48.145564+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:41:57.706735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"25'5","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.146032+0000","last_change":"2024-04-22T06:31:06.004971+0000","last_active":"2024-04-22T06:31:48.146032+0000","last_peered":"2024-04-22T06:31:48.146032+0000","last_clean":"2024-04-22T06:31:48.146032+0000","last_became_active":"2024-04-22T06:31:06.004473+0000","last_became_peered":"2024-04-22T06:31:06.004473+0000","last_unstale":"2024-04-22T06:31:48.146032+0000","last_undegraded":"2024-04-22T06:31:48.146032+0000","last_fullsized":"2024-04-22T06:31:48.146032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:04.971878+0000","last_clean_scrub_stamp":"2024-04-22T06:31:04.971878+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:47:11.735311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.191136+0000","last_change":"2024-04-22T06:31:07.018314+0000","last_active":"2024-04-22T06:31:48.191136+0000","last_peered":"2024-04-22T06:31:48.191136+0000","last_clean":"2024-04-22T06:31:48.191136+0000","last_became_active":"2024-04-22T06:31:07.018033+0000","last_became_peered":"2024-04-22T06:31:07.018033+0000","last_unstale":"2024-04-22T06:31:48.191136+0000","last_undegraded":"2024-04-22T06:31:48.191136+0000","last_fullsized":"2024-04-22T06:31:48.191136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:06:06.099494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.192586+0000","last_change":"2024-04-22T06:31:48.192586+0000","last_active":"2024-04-22T06:31:48.192586+0000","last_peered":"2024-04-22T06:31:48.192586+0000","last_clean":"2024-04-22T06:31:48.192586+0000","last_became_active":"2024-04-22T06:31:48.192127+0000","last_became_peered":"2024-04-22T06:31:48.192127+0000","last_unstale":"2024-04-22T06:31:48.192586+0000","last_undegraded":"2024-04-22T06:31:48.192586+0000","last_fullsized":"2024-04-22T06:31:48.192586+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:10.019817+0000","last_clean_scrub_stamp":"2024-04-22T06:31:10.019817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:58:18.053308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4,2],"acting":[3,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.144952+0000","last_change":"2024-04-22T06:31:07.024780+0000","last_active":"2024-04-22T06:31:48.144952+0000","last_peered":"2024-04-22T06:31:48.144952+0000","last_clean":"2024-04-22T06:31:48.144952+0000","last_became_active":"2024-04-22T06:31:07.024501+0000","last_became_peered":"2024-04-22T06:31:07.024501+0000","last_unstale":"2024-04-22T06:31:48.144952+0000","last_undegraded":"2024-04-22T06:31:48.144952+0000","last_fullsized":"2024-04-22T06:31:48.144952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:12:45.981232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.150411+0000","last_change":"2024-04-22T06:31:48.150411+0000","last_active":"2024-04-22T06:31:48.150411+0000","last_peered":"2024-04-22T06:31:48.150411+0000","last_clean":"2024-04-22T06:31:48.150411+0000","last_became_active":"2024-04-22T06:31:48.150131+0000","last_became_peered":"2024-04-22T06:31:48.150131+0000","last_unstale":"2024-04-22T06:31:48.150411+0000","last_undegraded":"2024-04-22T06:31:48.150411+0000","last_fullsized":"2024-04-22T06:31:48.150411+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:32:05.456771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143270+0000","last_change":"2024-04-22T06:31:07.018563+0000","last_active":"2024-04-22T06:31:48.143270+0000","last_peered":"2024-04-22T06:31:48.143270+0000","last_clean":"2024-04-22T06:31:48.143270+0000","last_became_active":"2024-04-22T06:31:07.018289+0000","last_became_peered":"2024-04-22T06:31:07.018289+0000","last_unstale":"2024-04-22T06:31:48.143270+0000","last_undegraded":"2024-04-22T06:31:48.143270+0000","last_fullsized":"2024-04-22T06:31:48.143270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:30:11.892544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142888+0000","last_change":"2024-04-22T06:31:07.022901+0000","last_active":"2024-04-22T06:31:48.142888+0000","last_peered":"2024-04-22T06:31:48.142888+0000","last_clean":"2024-04-22T06:31:48.142888+0000","last_became_active":"2024-04-22T06:31:07.022420+0000","last_became_peered":"2024-04-22T06:31:07.022420+0000","last_unstale":"2024-04-22T06:31:48.142888+0000","last_undegraded":"2024-04-22T06:31:48.142888+0000","last_fullsized":"2024-04-22T06:31:48.142888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:24:18.929896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.146584+0000","last_change":"2024-04-22T06:31:07.022875+0000","last_active":"2024-04-22T06:31:48.146584+0000","last_peered":"2024-04-22T06:31:48.146584+0000","last_clean":"2024-04-22T06:31:48.146584+0000","last_became_active":"2024-04-22T06:31:07.022545+0000","last_became_peered":"2024-04-22T06:31:07.022545+0000","last_unstale":"2024-04-22T06:31:48.146584+0000","last_undegraded":"2024-04-22T06:31:48.146584+0000","last_fullsized":"2024-04-22T06:31:48.146584+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:02:30.464508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.147729+0000","last_change":"2024-04-22T06:31:07.026367+0000","last_active":"2024-04-22T06:31:48.147729+0000","last_peered":"2024-04-22T06:31:48.147729+0000","last_clean":"2024-04-22T06:31:48.147729+0000","last_became_active":"2024-04-22T06:31:07.026085+0000","last_became_peered":"2024-04-22T06:31:07.026085+0000","last_unstale":"2024-04-22T06:31:48.147729+0000","last_undegraded":"2024-04-22T06:31:48.147729+0000","last_fullsized":"2024-04-22T06:31:48.147729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:02:55.368591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145845+0000","last_change":"2024-04-22T06:31:07.022989+0000","last_active":"2024-04-22T06:31:48.145845+0000","last_peered":"2024-04-22T06:31:48.145845+0000","last_clean":"2024-04-22T06:31:48.145845+0000","last_became_active":"2024-04-22T06:31:07.022688+0000","last_became_peered":"2024-04-22T06:31:07.022688+0000","last_unstale":"2024-04-22T06:31:48.145845+0000","last_undegraded":"2024-04-22T06:31:48.145845+0000","last_fullsized":"2024-04-22T06:31:48.145845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:24:10.910731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.146336+0000","last_change":"2024-04-22T06:31:07.026315+0000","last_active":"2024-04-22T06:31:48.146336+0000","last_peered":"2024-04-22T06:31:48.146336+0000","last_clean":"2024-04-22T06:31:48.146336+0000","last_became_active":"2024-04-22T06:31:07.025951+0000","last_became_peered":"2024-04-22T06:31:07.025951+0000","last_unstale":"2024-04-22T06:31:48.146336+0000","last_undegraded":"2024-04-22T06:31:48.146336+0000","last_fullsized":"2024-04-22T06:31:48.146336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:54:12.160058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142093+0000","last_change":"2024-04-22T06:31:07.022748+0000","last_active":"2024-04-22T06:31:48.142093+0000","last_peered":"2024-04-22T06:31:48.142093+0000","last_clean":"2024-04-22T06:31:48.142093+0000","last_became_active":"2024-04-22T06:31:07.022481+0000","last_became_peered":"2024-04-22T06:31:07.022481+0000","last_unstale":"2024-04-22T06:31:48.142093+0000","last_undegraded":"2024-04-22T06:31:48.142093+0000","last_fullsized":"2024-04-22T06:31:48.142093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:56:32.631637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145948+0000","last_change":"2024-04-22T06:31:07.021939+0000","last_active":"2024-04-22T06:31:48.145948+0000","last_peered":"2024-04-22T06:31:48.145948+0000","last_clean":"2024-04-22T06:31:48.145948+0000","last_became_active":"2024-04-22T06:31:07.021638+0000","last_became_peered":"2024-04-22T06:31:07.021638+0000","last_unstale":"2024-04-22T06:31:48.145948+0000","last_undegraded":"2024-04-22T06:31:48.145948+0000","last_fullsized":"2024-04-22T06:31:48.145948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:05:27.106092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143165+0000","last_change":"2024-04-22T06:31:07.015844+0000","last_active":"2024-04-22T06:31:48.143165+0000","last_peered":"2024-04-22T06:31:48.143165+0000","last_clean":"2024-04-22T06:31:48.143165+0000","last_became_active":"2024-04-22T06:31:07.015520+0000","last_became_peered":"2024-04-22T06:31:07.015520+0000","last_unstale":"2024-04-22T06:31:48.143165+0000","last_undegraded":"2024-04-22T06:31:48.143165+0000","last_fullsized":"2024-04-22T06:31:48.143165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:24:04.676892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142694+0000","last_change":"2024-04-22T06:31:07.023697+0000","last_active":"2024-04-22T06:31:48.142694+0000","last_peered":"2024-04-22T06:31:48.142694+0000","last_clean":"2024-04-22T06:31:48.142694+0000","last_became_active":"2024-04-22T06:31:07.023283+0000","last_became_peered":"2024-04-22T06:31:07.023283+0000","last_unstale":"2024-04-22T06:31:48.142694+0000","last_undegraded":"2024-04-22T06:31:48.142694+0000","last_fullsized":"2024-04-22T06:31:48.142694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:43:01.338104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145399+0000","last_change":"2024-04-22T06:31:07.023953+0000","last_active":"2024-04-22T06:31:48.145399+0000","last_peered":"2024-04-22T06:31:48.145399+0000","last_clean":"2024-04-22T06:31:48.145399+0000","last_became_active":"2024-04-22T06:31:07.023704+0000","last_became_peered":"2024-04-22T06:31:07.023704+0000","last_unstale":"2024-04-22T06:31:48.145399+0000","last_undegraded":"2024-04-22T06:31:48.145399+0000","last_fullsized":"2024-04-22T06:31:48.145399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:59:32.219492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142410+0000","last_change":"2024-04-22T06:31:07.027418+0000","last_active":"2024-04-22T06:31:48.142410+0000","last_peered":"2024-04-22T06:31:48.142410+0000","last_clean":"2024-04-22T06:31:48.142410+0000","last_became_active":"2024-04-22T06:31:07.027113+0000","last_became_peered":"2024-04-22T06:31:07.027113+0000","last_unstale":"2024-04-22T06:31:48.142410+0000","last_undegraded":"2024-04-22T06:31:48.142410+0000","last_fullsized":"2024-04-22T06:31:48.142410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:08:42.204674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145946+0000","last_change":"2024-04-22T06:31:07.015520+0000","last_active":"2024-04-22T06:31:48.145946+0000","last_peered":"2024-04-22T06:31:48.145946+0000","last_clean":"2024-04-22T06:31:48.145946+0000","last_became_active":"2024-04-22T06:31:07.015199+0000","last_became_peered":"2024-04-22T06:31:07.015199+0000","last_unstale":"2024-04-22T06:31:48.145946+0000","last_undegraded":"2024-04-22T06:31:48.145946+0000","last_fullsized":"2024-04-22T06:31:48.145946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:48:40.688931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142835+0000","last_change":"2024-04-22T06:31:07.016457+0000","last_active":"2024-04-22T06:31:48.142835+0000","last_peered":"2024-04-22T06:31:48.142835+0000","last_clean":"2024-04-22T06:31:48.142835+0000","last_became_active":"2024-04-22T06:31:07.016082+0000","last_became_peered":"2024-04-22T06:31:07.016082+0000","last_unstale":"2024-04-22T06:31:48.142835+0000","last_undegraded":"2024-04-22T06:31:48.142835+0000","last_fullsized":"2024-04-22T06:31:48.142835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:12:28.517065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.141985+0000","last_change":"2024-04-22T06:31:07.019955+0000","last_active":"2024-04-22T06:31:48.141985+0000","last_peered":"2024-04-22T06:31:48.141985+0000","last_clean":"2024-04-22T06:31:48.141985+0000","last_became_active":"2024-04-22T06:31:07.019174+0000","last_became_peered":"2024-04-22T06:31:07.019174+0000","last_unstale":"2024-04-22T06:31:48.141985+0000","last_undegraded":"2024-04-22T06:31:48.141985+0000","last_fullsized":"2024-04-22T06:31:48.141985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:09:07.106161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.147563+0000","last_change":"2024-04-22T06:31:07.019341+0000","last_active":"2024-04-22T06:31:48.147563+0000","last_peered":"2024-04-22T06:31:48.147563+0000","last_clean":"2024-04-22T06:31:48.147563+0000","last_became_active":"2024-04-22T06:31:07.019048+0000","last_became_peered":"2024-04-22T06:31:07.019048+0000","last_unstale":"2024-04-22T06:31:48.147563+0000","last_undegraded":"2024-04-22T06:31:48.147563+0000","last_fullsized":"2024-04-22T06:31:48.147563+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:04:19.362686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143194+0000","last_change":"2024-04-22T06:31:07.019433+0000","last_active":"2024-04-22T06:31:48.143194+0000","last_peered":"2024-04-22T06:31:48.143194+0000","last_clean":"2024-04-22T06:31:48.143194+0000","last_became_active":"2024-04-22T06:31:07.019133+0000","last_became_peered":"2024-04-22T06:31:07.019133+0000","last_unstale":"2024-04-22T06:31:48.143194+0000","last_undegraded":"2024-04-22T06:31:48.143194+0000","last_fullsized":"2024-04-22T06:31:48.143194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:01:54.466486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145847+0000","last_change":"2024-04-22T06:31:07.023014+0000","last_active":"2024-04-22T06:31:48.145847+0000","last_peered":"2024-04-22T06:31:48.145847+0000","last_clean":"2024-04-22T06:31:48.145847+0000","last_became_active":"2024-04-22T06:31:07.022716+0000","last_became_peered":"2024-04-22T06:31:07.022716+0000","last_unstale":"2024-04-22T06:31:48.145847+0000","last_undegraded":"2024-04-22T06:31:48.145847+0000","last_fullsized":"2024-04-22T06:31:48.145847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:11:47.309706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.191327+0000","last_change":"2024-04-22T06:31:07.020446+0000","last_active":"2024-04-22T06:31:48.191327+0000","last_peered":"2024-04-22T06:31:48.191327+0000","last_clean":"2024-04-22T06:31:48.191327+0000","last_became_active":"2024-04-22T06:31:07.018995+0000","last_became_peered":"2024-04-22T06:31:07.018995+0000","last_unstale":"2024-04-22T06:31:48.191327+0000","last_undegraded":"2024-04-22T06:31:48.191327+0000","last_fullsized":"2024-04-22T06:31:48.191327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:56:45.031262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.149688+0000","last_change":"2024-04-22T06:31:48.149688+0000","last_active":"2024-04-22T06:31:48.149688+0000","last_peered":"2024-04-22T06:31:48.149688+0000","last_clean":"2024-04-22T06:31:48.149688+0000","last_became_active":"2024-04-22T06:31:48.149398+0000","last_became_peered":"2024-04-22T06:31:48.149398+0000","last_unstale":"2024-04-22T06:31:48.149688+0000","last_undegraded":"2024-04-22T06:31:48.149688+0000","last_fullsized":"2024-04-22T06:31:48.149688+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:55:49.846070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145780+0000","last_change":"2024-04-22T06:31:07.014209+0000","last_active":"2024-04-22T06:31:48.145780+0000","last_peered":"2024-04-22T06:31:48.145780+0000","last_clean":"2024-04-22T06:31:48.145780+0000","last_became_active":"2024-04-22T06:31:07.013946+0000","last_became_peered":"2024-04-22T06:31:07.013946+0000","last_unstale":"2024-04-22T06:31:48.145780+0000","last_undegraded":"2024-04-22T06:31:48.145780+0000","last_fullsized":"2024-04-22T06:31:48.145780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:40:31.669158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143602+0000","last_change":"2024-04-22T06:31:07.008764+0000","last_active":"2024-04-22T06:31:48.143602+0000","last_peered":"2024-04-22T06:31:48.143602+0000","last_clean":"2024-04-22T06:31:48.143602+0000","last_became_active":"2024-04-22T06:31:07.007810+0000","last_became_peered":"2024-04-22T06:31:07.007810+0000","last_unstale":"2024-04-22T06:31:48.143602+0000","last_undegraded":"2024-04-22T06:31:48.143602+0000","last_fullsized":"2024-04-22T06:31:48.143602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:49:54.894818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.145136+0000","last_change":"2024-04-22T06:31:07.022141+0000","last_active":"2024-04-22T06:31:48.145136+0000","last_peered":"2024-04-22T06:31:48.145136+0000","last_clean":"2024-04-22T06:31:48.145136+0000","last_became_active":"2024-04-22T06:31:07.021845+0000","last_became_peered":"2024-04-22T06:31:07.021845+0000","last_unstale":"2024-04-22T06:31:48.145136+0000","last_undegraded":"2024-04-22T06:31:48.145136+0000","last_fullsized":"2024-04-22T06:31:48.145136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:26:28.598104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142294+0000","last_change":"2024-04-22T06:31:07.012188+0000","last_active":"2024-04-22T06:31:48.142294+0000","last_peered":"2024-04-22T06:31:48.142294+0000","last_clean":"2024-04-22T06:31:48.142294+0000","last_became_active":"2024-04-22T06:31:07.011909+0000","last_became_peered":"2024-04-22T06:31:07.011909+0000","last_unstale":"2024-04-22T06:31:48.142294+0000","last_undegraded":"2024-04-22T06:31:48.142294+0000","last_fullsized":"2024-04-22T06:31:48.142294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:25:13.089928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143079+0000","last_change":"2024-04-22T06:31:07.020033+0000","last_active":"2024-04-22T06:31:48.143079+0000","last_peered":"2024-04-22T06:31:48.143079+0000","last_clean":"2024-04-22T06:31:48.143079+0000","last_became_active":"2024-04-22T06:31:07.019689+0000","last_became_peered":"2024-04-22T06:31:07.019689+0000","last_unstale":"2024-04-22T06:31:48.143079+0000","last_undegraded":"2024-04-22T06:31:48.143079+0000","last_fullsized":"2024-04-22T06:31:48.143079+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:09:07.209203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142885+0000","last_change":"2024-04-22T06:31:07.027287+0000","last_active":"2024-04-22T06:31:48.142885+0000","last_peered":"2024-04-22T06:31:48.142885+0000","last_clean":"2024-04-22T06:31:48.142885+0000","last_became_active":"2024-04-22T06:31:07.027015+0000","last_became_peered":"2024-04-22T06:31:07.027015+0000","last_unstale":"2024-04-22T06:31:48.142885+0000","last_undegraded":"2024-04-22T06:31:48.142885+0000","last_fullsized":"2024-04-22T06:31:48.142885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:42:49.322413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143284+0000","last_change":"2024-04-22T06:31:07.026332+0000","last_active":"2024-04-22T06:31:48.143284+0000","last_peered":"2024-04-22T06:31:48.143284+0000","last_clean":"2024-04-22T06:31:48.143284+0000","last_became_active":"2024-04-22T06:31:07.026018+0000","last_became_peered":"2024-04-22T06:31:07.026018+0000","last_unstale":"2024-04-22T06:31:48.143284+0000","last_undegraded":"2024-04-22T06:31:48.143284+0000","last_fullsized":"2024-04-22T06:31:48.143284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:27:08.560056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.143261+0000","last_change":"2024-04-22T06:31:07.020497+0000","last_active":"2024-04-22T06:31:48.143261+0000","last_peered":"2024-04-22T06:31:48.143261+0000","last_clean":"2024-04-22T06:31:48.143261+0000","last_became_active":"2024-04-22T06:31:07.020194+0000","last_became_peered":"2024-04-22T06:31:07.020194+0000","last_unstale":"2024-04-22T06:31:48.143261+0000","last_undegraded":"2024-04-22T06:31:48.143261+0000","last_fullsized":"2024-04-22T06:31:48.143261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:56:59.765295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.148064+0000","last_change":"2024-04-22T06:31:07.018396+0000","last_active":"2024-04-22T06:31:48.148064+0000","last_peered":"2024-04-22T06:31:48.148064+0000","last_clean":"2024-04-22T06:31:48.148064+0000","last_became_active":"2024-04-22T06:31:07.018073+0000","last_became_peered":"2024-04-22T06:31:07.018073+0000","last_unstale":"2024-04-22T06:31:48.148064+0000","last_undegraded":"2024-04-22T06:31:48.148064+0000","last_fullsized":"2024-04-22T06:31:48.148064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:37:06.501499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-22T06:31:48.142711+0000","last_change":"2024-04-22T06:31:07.014439+0000","last_active":"2024-04-22T06:31:48.142711+0000","last_peered":"2024-04-22T06:31:48.142711+0000","last_clean":"2024-04-22T06:31:48.142711+0000","last_became_active":"2024-04-22T06:31:07.014124+0000","last_became_peered":"2024-04-22T06:31:07.014124+0000","last_unstale":"2024-04-22T06:31:48.142711+0000","last_undegraded":"2024-04-22T06:31:48.142711+0000","last_fullsized":"2024-04-22T06:31:48.142711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T06:31:05.978695+0000","last_clean_scrub_stamp":"2024-04-22T06:31:05.978695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:58:46.075068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":10264,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":208,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":20528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":182,"ondisk_log_size":182,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32076,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339764,"statfs":{"total":96636764160,"available":96603918336,"internally_reserved":0,"allocated":921600,"data_stored":711508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31488,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340352,"statfs":{"total":96636764160,"available":96604520448,"internally_reserved":0,"allocated":319488,"data_stored":123672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":339968,"data_stored":122972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31484,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340356,"statfs":{"total":96636764160,"available":96604524544,"internally_reserved":0,"allocated":315392,"data_stored":120370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":323584,"data_stored":123706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":323584,"data_stored":120502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":319488,"data_stored":120572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343932,"statfs":{"total":96636764160,"available":96608186368,"internally_reserved":0,"allocated":913408,"data_stored":710282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T06:32:08.481 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T06:32:08.613 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T06:32:08.615 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T06:32:08.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.619+0000 7f92c2d4a640 1 Processor -- start 2024-04-22T06:32:08.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.619+0000 7f92c2d4a640 1 -- start start 2024-04-22T06:32:08.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.620+0000 7f92c2d4a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f92bc15b990 0x7f92bc0c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:08.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.620+0000 7f92c2d4a640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92bc15bd70 0x7f92bc15b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:08.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.620+0000 7f92c2d4a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92bc0c8e90 con 0x7f92bc0c78a0 2024-04-22T06:32:08.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.620+0000 7f92c2d4a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f92bc15ed90 con 0x7f92bc15b990 2024-04-22T06:32:08.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.620+0000 7f92c2d4a640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f92bc0c9130 con 0x7f92bc15bd70 2024-04-22T06:32:08.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.620+0000 7f92c12e7640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92bc15bd70 0x7f92bc15b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:08.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.620+0000 7f92c0ae6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f92bc15b990 0x7f92bc0c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:08.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.620+0000 7f92c12e7640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92bc15bd70 0x7f92bc15b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.44:42158/0 (socket says 172.21.15.44:42158) 2024-04-22T06:32:08.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.620+0000 7f92c12e7640 1 -- 172.21.15.44:0/3394659342 learned_addr learned my addr 172.21.15.44:0/3394659342 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:08.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.621+0000 7f92c12e7640 1 -- 172.21.15.44:0/3394659342 >> v2:172.21.15.70:3300/0 conn(0x7f92bc15b990 msgr2=0x7f92bc0c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:08.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.621+0000 7f92c12e7640 1 --2- 172.21.15.44:0/3394659342 >> v2:172.21.15.70:3300/0 conn(0x7f92bc15b990 0x7f92bc0c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:08.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.621+0000 7f92c12e7640 1 -- 172.21.15.44:0/3394659342 >> v1:172.21.15.70:6789/0 conn(0x7f92bc0c78a0 legacy=0x7f92bc06fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:08.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.621+0000 7f92c12e7640 1 -- 172.21.15.44:0/3394659342 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92bc0c93f0 con 0x7f92bc15bd70 2024-04-22T06:32:08.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.621+0000 7f92c0ae6640 1 --2- 172.21.15.44:0/3394659342 >> v2:172.21.15.70:3300/0 conn(0x7f92bc15b990 0x7f92bc0c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:32:08.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.622+0000 7f92c12e7640 1 --2- 172.21.15.44:0/3394659342 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92bc15bd70 0x7f92bc15b0b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f92b4009e50 tx=0x7f92b402bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f58bba41fab76b15 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:08.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.622+0000 7f92b37fe640 1 -- 172.21.15.44:0/3394659342 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b402cb00 con 0x7f92bc15bd70 2024-04-22T06:32:08.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.622+0000 7f92b37fe640 1 -- 172.21.15.44:0/3394659342 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92b402cca0 con 0x7f92bc15bd70 2024-04-22T06:32:08.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.622+0000 7f92b37fe640 1 -- 172.21.15.44:0/3394659342 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b4034610 con 0x7f92bc15bd70 2024-04-22T06:32:08.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.622+0000 7f92c2d4a640 1 -- 172.21.15.44:0/3394659342 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92bc15bd70 msgr2=0x7f92bc15b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:08.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.622+0000 7f92c2d4a640 1 --2- 172.21.15.44:0/3394659342 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92bc15bd70 0x7f92bc15b0b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f92b4009e50 tx=0x7f92b402bf60 comp rx=0 tx=0).stop 2024-04-22T06:32:08.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.623+0000 7f92c2d4a640 1 -- 172.21.15.44:0/3394659342 shutdown_connections 2024-04-22T06:32:08.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.623+0000 7f92c2d4a640 1 --2- 172.21.15.44:0/3394659342 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92bc15bd70 0x7f92bc15b0b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:08.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.623+0000 7f92c2d4a640 1 --2- 172.21.15.44:0/3394659342 >> v2:172.21.15.70:3300/0 conn(0x7f92bc15b990 0x7f92bc0c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:08.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.623+0000 7f92c2d4a640 1 -- 172.21.15.44:0/3394659342 >> 172.21.15.44:0/3394659342 conn(0x7f92bc0c9fd0 msgr2=0x7f92bc06f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:08.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.623+0000 7f92c2d4a640 1 -- 172.21.15.44:0/3394659342 shutdown_connections 2024-04-22T06:32:08.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.623+0000 7f92c2d4a640 1 -- 172.21.15.44:0/3394659342 wait complete. 2024-04-22T06:32:08.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.624+0000 7f92c2d4a640 1 Processor -- start 2024-04-22T06:32:08.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.624+0000 7f92c2d4a640 1 -- start start 2024-04-22T06:32:08.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.625+0000 7f92c2d4a640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92bc0c78a0 0x7f92bc06d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:08.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.625+0000 7f92c2d4a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92bc15b990 0x7f92bc15de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:08.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.625+0000 7f92c2d4a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92bc15bd70 0x7f92bc1f3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:08.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.625+0000 7f92c2d4a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f92bc0c96d0 con 0x7f92bc15b990 2024-04-22T06:32:08.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.625+0000 7f92c2d4a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f92bc15ea20 con 0x7f92bc15bd70 2024-04-22T06:32:08.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.625+0000 7f92c2d4a640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f92bc070120 con 0x7f92bc0c78a0 2024-04-22T06:32:08.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.625+0000 7f92b3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92bc15b990 0x7f92bc15de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:08.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.625+0000 7f92b3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92bc15b990 0x7f92bc15de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52540/0 (socket says 172.21.15.44:52540) 2024-04-22T06:32:08.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.625+0000 7f92b3fff640 1 -- 172.21.15.44:0/1786423367 learned_addr learned my addr 172.21.15.44:0/1786423367 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:08.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.625+0000 7f92c0ae6640 1 --2- 172.21.15.44:0/1786423367 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92bc0c78a0 0x7f92bc06d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:08.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.625+0000 7f92c12e7640 1 --2- 172.21.15.44:0/1786423367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92bc15bd70 0x7f92bc1f3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:08.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.625+0000 7f92b3fff640 1 -- 172.21.15.44:0/1786423367 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92bc0c78a0 msgr2=0x7f92bc06d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:08.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.625+0000 7f92b3fff640 1 --2- 172.21.15.44:0/1786423367 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92bc0c78a0 0x7f92bc06d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:08.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.625+0000 7f92b3fff640 1 -- 172.21.15.44:0/1786423367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92bc15bd70 msgr2=0x7f92bc1f3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:08.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.625+0000 7f92b3fff640 1 --2- 172.21.15.44:0/1786423367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92bc15bd70 0x7f92bc1f3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:08.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.625+0000 7f92b3fff640 1 -- 172.21.15.44:0/1786423367 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92bc15afb0 con 0x7f92bc15b990 2024-04-22T06:32:08.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.626+0000 7f92b3fff640 1 --2- 172.21.15.44:0/1786423367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92bc15b990 0x7f92bc15de00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f92ac00c3f0 tx=0x7f92ac00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:08.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.626+0000 7f92b17fa640 1 -- 172.21.15.44:0/1786423367 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ac0052d0 con 0x7f92bc15b990 2024-04-22T06:32:08.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.626+0000 7f92b17fa640 1 -- 172.21.15.44:0/1786423367 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92ac005490 con 0x7f92bc15b990 2024-04-22T06:32:08.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.626+0000 7f92c2d4a640 1 -- 172.21.15.44:0/1786423367 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92bc15b290 con 0x7f92bc15b990 2024-04-22T06:32:08.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.626+0000 7f92b17fa640 1 -- 172.21.15.44:0/1786423367 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ac007260 con 0x7f92bc15b990 2024-04-22T06:32:08.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.626+0000 7f92c2d4a640 1 -- 172.21.15.44:0/1786423367 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92bc1e60f0 con 0x7f92bc15b990 2024-04-22T06:32:08.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.627+0000 7f92c2d4a640 1 -- 172.21.15.44:0/1786423367 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92bc069310 con 0x7f92bc15b990 2024-04-22T06:32:08.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.628+0000 7f92b17fa640 1 -- 172.21.15.44:0/1786423367 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f92ac007400 con 0x7f92bc15b990 2024-04-22T06:32:08.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.628+0000 7f92b17fa640 1 --2- 172.21.15.44:0/1786423367 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f92a806c8f0 0x7f92a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:08.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.628+0000 7f92b17fa640 1 -- 172.21.15.44:0/1786423367 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f92ac08b3c0 con 0x7f92bc15b990 2024-04-22T06:32:08.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.628+0000 7f92c0ae6640 1 --2- 172.21.15.44:0/1786423367 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f92a806c8f0 0x7f92a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:08.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.629+0000 7f92c0ae6640 1 --2- 172.21.15.44:0/1786423367 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f92a806c8f0 0x7f92a806edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f92b8009560 tx=0x7f92b8009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:08.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.633+0000 7f92b17fa640 1 -- 172.21.15.44:0/1786423367 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f92ac058d20 con 0x7f92bc15b990 2024-04-22T06:32:08.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.860+0000 7f92c2d4a640 1 -- 172.21.15.44:0/1786423367 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92bc0c2e00 con 0x7f92bc15b990 2024-04-22T06:32:08.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.860+0000 7f92b17fa640 1 -- 172.21.15.44:0/1786423367 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+6905 (secure 0 0 0) 0x7f92ac05f020 con 0x7f92bc15b990 2024-04-22T06:32:08.859 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:32:08.860 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/1779271860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":1779271860},{"type":"v1","addr":"172.21.15.70:6836","nonce":1779271860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T06:31:06.356312+0000","modified":"2024-04-22T06:31:56.168744+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4308,"mds_1":4305,"mds_2":4293,"mds_3":4290,"mds_4":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":4,"incarnation":17,"state":"up:active","state_seq":15,"addr":"172.21.15.44:6834/1497302315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1497302315},{"type":"v1","addr":"172.21.15.44:6834","nonce":1497302315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4290":{"gid":4290,"name":"a","rank":3,"incarnation":15,"state":"up:active","state_seq":14,"addr":"172.21.15.44:6835/2383621014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2383621014},{"type":"v1","addr":"172.21.15.44:6835","nonce":2383621014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"e","rank":2,"incarnation":13,"state":"up:active","state_seq":12,"addr":"172.21.15.44:6837/915960587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":915960587},{"type":"v1","addr":"172.21.15.44:6837","nonce":915960587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4305":{"gid":4305,"name":"b","rank":1,"incarnation":9,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6838/924955785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":924955785},{"type":"v1","addr":"172.21.15.70:6838","nonce":924955785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4308":{"gid":4308,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.70:6839/2915150040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6837","nonce":2915150040},{"type":"v1","addr":"172.21.15.70:6839","nonce":2915150040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4308,"qdb_cluster":[4287,4290,4293,4305,4308]},"id":1}]} 2024-04-22T06:32:08.860 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 18 2024-04-22T06:32:08.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.863+0000 7f92c2d4a640 1 -- 172.21.15.44:0/1786423367 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f92a806c8f0 msgr2=0x7f92a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:08.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.863+0000 7f92c2d4a640 1 --2- 172.21.15.44:0/1786423367 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f92a806c8f0 0x7f92a806edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f92b8009560 tx=0x7f92b8009190 comp rx=0 tx=0).stop 2024-04-22T06:32:08.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.863+0000 7f92c2d4a640 1 -- 172.21.15.44:0/1786423367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92bc15b990 msgr2=0x7f92bc15de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:08.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.863+0000 7f92c2d4a640 1 --2- 172.21.15.44:0/1786423367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92bc15b990 0x7f92bc15de00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f92ac00c3f0 tx=0x7f92ac00c8b0 comp rx=0 tx=0).stop 2024-04-22T06:32:08.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.863+0000 7f92c2d4a640 1 -- 172.21.15.44:0/1786423367 shutdown_connections 2024-04-22T06:32:08.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.863+0000 7f92c2d4a640 1 --2- 172.21.15.44:0/1786423367 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f92a806c8f0 0x7f92a806edb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:08.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.863+0000 7f92c2d4a640 1 --2- 172.21.15.44:0/1786423367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92bc15bd70 0x7f92bc1f3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:08.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.863+0000 7f92c2d4a640 1 --2- 172.21.15.44:0/1786423367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92bc15b990 0x7f92bc15de00 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:08.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.863+0000 7f92c2d4a640 1 --2- 172.21.15.44:0/1786423367 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92bc0c78a0 0x7f92bc06d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:08.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.863+0000 7f92c2d4a640 1 -- 172.21.15.44:0/1786423367 >> 172.21.15.44:0/1786423367 conn(0x7f92bc0c9fd0 msgr2=0x7f92bc0cb360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:08.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.863+0000 7f92c2d4a640 1 -- 172.21.15.44:0/1786423367 shutdown_connections 2024-04-22T06:32:08.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.863+0000 7f92c2d4a640 1 -- 172.21.15.44:0/1786423367 wait complete. 2024-04-22T06:32:08.873 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T06:32:08.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.974+0000 7fdd97a77640 1 Processor -- start 2024-04-22T06:32:08.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.975+0000 7fdd97a77640 1 -- start start 2024-04-22T06:32:08.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.975+0000 7fdd97a77640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdd9015e670 0x7fdd9015ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:08.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.975+0000 7fdd97a77640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd900cb990 con 0x7fdd900c7d00 2024-04-22T06:32:08.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.975+0000 7fdd97a77640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd900cc530 con 0x7fdd900cd170 2024-04-22T06:32:08.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.975+0000 7fdd97a77640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fdd9015f480 con 0x7fdd9015e670 2024-04-22T06:32:08.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.976+0000 7fdd95012640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fdd900c7d00 0x7fdd9006d030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40176/0 (socket says 172.21.15.44:40176) 2024-04-22T06:32:08.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.976+0000 7fdd95012640 1 -- 172.21.15.44:0/2420154917 learned_addr learned my addr 172.21.15.44:0/2420154917 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:32:08.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.976+0000 7fdd95813640 1 --2- 172.21.15.44:0/2420154917 >> v2:172.21.15.70:3300/0 conn(0x7fdd9015e670 0x7fdd9015ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:08.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.976+0000 7fdd94811640 1 -- 172.21.15.44:0/2420154917 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 101989124 0 0) 0x7fdd900cb990 con 0x7fdd900c7d00 2024-04-22T06:32:08.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.976+0000 7fdd94811640 1 -- 172.21.15.44:0/2420154917 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd7c003520 con 0x7fdd900c7d00 2024-04-22T06:32:08.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.976+0000 7fdd94811640 1 -- 172.21.15.44:0/2420154917 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3030531568 0 0) 0x7fdd7c003520 con 0x7fdd900c7d00 2024-04-22T06:32:08.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.976+0000 7fdd94811640 1 -- 172.21.15.44:0/2420154917 >> v2:172.21.15.70:3300/0 conn(0x7fdd9015e670 msgr2=0x7fdd9015ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:08.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.976+0000 7fdd94811640 1 --2- 172.21.15.44:0/2420154917 >> v2:172.21.15.70:3300/0 conn(0x7fdd9015e670 0x7fdd9015ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:08.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.976+0000 7fdd94811640 1 -- 172.21.15.44:0/2420154917 >> v1:172.21.15.70:6789/0 conn(0x7fdd900cd170 legacy=0x7fdd900cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:08.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.976+0000 7fdd94811640 1 -- 172.21.15.44:0/2420154917 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd900cc810 con 0x7fdd900c7d00 2024-04-22T06:32:08.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.976+0000 7fdd94811640 1 -- 172.21.15.44:0/2420154917 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1237339257 0 0) 0x7fdd80001670 con 0x7fdd900c7d00 2024-04-22T06:32:08.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.976+0000 7fdd94811640 1 -- 172.21.15.44:0/2420154917 >> v1:172.21.15.44:6789/0 conn(0x7fdd900c7d00 legacy=0x7fdd9006d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:08.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.976+0000 7fdd94811640 1 --2- 172.21.15.44:0/2420154917 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd7c0039b0 0x7fdd7c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:08.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.976+0000 7fdd94811640 1 --2- 172.21.15.44:0/2420154917 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdd7c004800 0x7fdd7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:08.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.976+0000 7fdd94811640 1 --2- 172.21.15.44:0/2420154917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd7c0051c0 0x7fdd7c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:08.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.976+0000 7fdd94811640 1 -- 172.21.15.44:0/2420154917 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdd7c00c130 con 0x7fdd7c0051c0 2024-04-22T06:32:08.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.976+0000 7fdd94811640 1 -- 172.21.15.44:0/2420154917 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fdd7c00c2e0 con 0x7fdd7c0039b0 2024-04-22T06:32:08.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.976+0000 7fdd94811640 1 -- 172.21.15.44:0/2420154917 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdd7c00c4e0 con 0x7fdd7c004800 2024-04-22T06:32:08.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.976+0000 7fdd96014640 1 --2- 172.21.15.44:0/2420154917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd7c0051c0 0x7fdd7c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:08.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.976+0000 7fdd95813640 1 --2- 172.21.15.44:0/2420154917 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd7c0039b0 0x7fdd7c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:08.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.976+0000 7fdd95012640 1 --2- 172.21.15.44:0/2420154917 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdd7c004800 0x7fdd7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:08.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.977+0000 7fdd96014640 1 -- 172.21.15.44:0/2420154917 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdd7c004800 msgr2=0x7fdd7c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:08.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.977+0000 7fdd96014640 1 --2- 172.21.15.44:0/2420154917 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdd7c004800 0x7fdd7c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:08.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.977+0000 7fdd96014640 1 -- 172.21.15.44:0/2420154917 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd7c0039b0 msgr2=0x7fdd7c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:08.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.977+0000 7fdd96014640 1 --2- 172.21.15.44:0/2420154917 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd7c0039b0 0x7fdd7c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:08.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.977+0000 7fdd96014640 1 -- 172.21.15.44:0/2420154917 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd7c003520 con 0x7fdd7c0051c0 2024-04-22T06:32:08.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.977+0000 7fdd96014640 1 -- 172.21.15.44:0/2420154917 reap_dead start 2024-04-22T06:32:08.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.977+0000 7fdd96014640 1 --2- 172.21.15.44:0/2420154917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd7c0051c0 0x7fdd7c00b9c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdd88001e50 tx=0x7fdd7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d700089aaa6d3ad5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:08.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.977+0000 7fdd94811640 1 -- 172.21.15.44:0/2420154917 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd88001a70 con 0x7fdd7c0051c0 2024-04-22T06:32:08.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.977+0000 7fdd94811640 1 -- 172.21.15.44:0/2420154917 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd88003bb0 con 0x7fdd7c0051c0 2024-04-22T06:32:08.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.977+0000 7fdd94811640 1 -- 172.21.15.44:0/2420154917 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd880050d0 con 0x7fdd7c0051c0 2024-04-22T06:32:08.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.978+0000 7fdd97a77640 1 -- 172.21.15.44:0/2420154917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd7c0051c0 msgr2=0x7fdd7c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:08.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.978+0000 7fdd97a77640 1 --2- 172.21.15.44:0/2420154917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd7c0051c0 0x7fdd7c00b9c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdd88001e50 tx=0x7fdd7c005830 comp rx=0 tx=0).stop 2024-04-22T06:32:08.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.978+0000 7fdd97a77640 1 -- 172.21.15.44:0/2420154917 shutdown_connections 2024-04-22T06:32:08.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.978+0000 7fdd97a77640 1 --2- 172.21.15.44:0/2420154917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd7c0051c0 0x7fdd7c00b9c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:08.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.978+0000 7fdd97a77640 1 -- 172.21.15.44:0/2420154917 >> 172.21.15.44:0/2420154917 conn(0x7fdd90072fb0 msgr2=0x7fdd900733c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:08.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.978+0000 7fdd97a77640 1 -- 172.21.15.44:0/2420154917 shutdown_connections 2024-04-22T06:32:08.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.978+0000 7fdd97a77640 1 -- 172.21.15.44:0/2420154917 wait complete. 2024-04-22T06:32:08.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.979+0000 7fdd97a77640 1 Processor -- start 2024-04-22T06:32:08.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.979+0000 7fdd97a77640 1 -- start start 2024-04-22T06:32:08.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.980+0000 7fdd97a77640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd7c0051c0 0x7fdd900cf830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:08.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.980+0000 7fdd97a77640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd900d0fa0 0x7fdd90156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:08.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.980+0000 7fdd97a77640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdd90157000 0x7fdd90148740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:08.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.980+0000 7fdd97a77640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdd900cbf70 con 0x7fdd7c0051c0 2024-04-22T06:32:08.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.980+0000 7fdd97a77640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fdd901558d0 con 0x7fdd900d0fa0 2024-04-22T06:32:08.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.980+0000 7fdd97a77640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdd900cbc00 con 0x7fdd90157000 2024-04-22T06:32:08.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.980+0000 7fdd95813640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd7c0051c0 0x7fdd900cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:08.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.980+0000 7fdd95813640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd7c0051c0 0x7fdd900cf830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52548/0 (socket says 172.21.15.44:52548) 2024-04-22T06:32:08.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.980+0000 7fdd95813640 1 -- 172.21.15.44:0/1095260379 learned_addr learned my addr 172.21.15.44:0/1095260379 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:08.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.980+0000 7fdd96014640 1 --2- 172.21.15.44:0/1095260379 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdd90157000 0x7fdd90148740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:08.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.980+0000 7fdd95012640 1 --2- 172.21.15.44:0/1095260379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd900d0fa0 0x7fdd90156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:08.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.980+0000 7fdd95813640 1 -- 172.21.15.44:0/1095260379 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdd90157000 msgr2=0x7fdd90148740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:08.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.980+0000 7fdd95813640 1 --2- 172.21.15.44:0/1095260379 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdd90157000 0x7fdd90148740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:08.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.980+0000 7fdd95813640 1 -- 172.21.15.44:0/1095260379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd900d0fa0 msgr2=0x7fdd90156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:08.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.980+0000 7fdd95813640 1 --2- 172.21.15.44:0/1095260379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd900d0fa0 0x7fdd90156ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:08.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.980+0000 7fdd95813640 1 -- 172.21.15.44:0/1095260379 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd900ce7d0 con 0x7fdd7c0051c0 2024-04-22T06:32:08.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.981+0000 7fdd95813640 1 --2- 172.21.15.44:0/1095260379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd7c0051c0 0x7fdd900cf830 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c011940 tx=0x7fdd8c011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:08.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.981+0000 7fdd867fc640 1 -- 172.21.15.44:0/1095260379 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c00faf0 con 0x7fdd7c0051c0 2024-04-22T06:32:08.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.981+0000 7fdd867fc640 1 -- 172.21.15.44:0/1095260379 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd8c00fc90 con 0x7fdd7c0051c0 2024-04-22T06:32:08.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.981+0000 7fdd97a77640 1 -- 172.21.15.44:0/1095260379 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd900c3e80 con 0x7fdd7c0051c0 2024-04-22T06:32:08.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.981+0000 7fdd867fc640 1 -- 172.21.15.44:0/1095260379 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c014440 con 0x7fdd7c0051c0 2024-04-22T06:32:08.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.981+0000 7fdd97a77640 1 -- 172.21.15.44:0/1095260379 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd900cea70 con 0x7fdd7c0051c0 2024-04-22T06:32:08.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.982+0000 7fdd97a77640 1 -- 172.21.15.44:0/1095260379 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd90068cc0 con 0x7fdd7c0051c0 2024-04-22T06:32:08.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.982+0000 7fdd867fc640 1 -- 172.21.15.44:0/1095260379 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdd8c012040 con 0x7fdd7c0051c0 2024-04-22T06:32:08.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.983+0000 7fdd867fc640 1 --2- 172.21.15.44:0/1095260379 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fdd7406c8f0 0x7fdd7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:08.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.983+0000 7fdd867fc640 1 -- 172.21.15.44:0/1095260379 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fdd8c08c430 con 0x7fdd7c0051c0 2024-04-22T06:32:08.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.983+0000 7fdd95012640 1 --2- 172.21.15.44:0/1095260379 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fdd7406c8f0 0x7fdd7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:08.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.984+0000 7fdd95012640 1 --2- 172.21.15.44:0/1095260379 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fdd7406c8f0 0x7fdd7406edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdd80001050 tx=0x7fdd80007040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:08.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:08.987+0000 7fdd867fc640 1 -- 172.21.15.44:0/1095260379 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdd8c019030 con 0x7fdd7c0051c0 2024-04-22T06:32:09.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.172+0000 7fdd97a77640 1 -- 172.21.15.44:0/1095260379 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdd900cb260 con 0x7fdd7c0051c0 2024-04-22T06:32:09.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.173+0000 7fdd867fc640 1 -- 172.21.15.44:0/1095260379 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18794 (secure 0 0 0) 0x7fdd8c059ea0 con 0x7fdd7c0051c0 2024-04-22T06:32:09.172 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:32:09.172 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":27,"fsid":"9b0a34ca-403d-4513-8a66-d3749ba559aa","created":"2024-04-22T06:30:42.334581+0000","modified":"2024-04-22T06:31:48.138913+0000","last_up_change":"2024-04-22T06:30:52.942257+0000","last_in_change":"2024-04-22T06:30:44.467727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T06:30:54.560256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T06:31:04.612387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T06:31:05.312124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-22T06:31:09.025550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b4a6692-53c9-472d-b2f3-1dd29a6679ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6817","nonce":2220268493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6819","nonce":2220268493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6823","nonce":2220268493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6821","nonce":2220268493}]},"public_addr":"172.21.15.44:6817/2220268493","cluster_addr":"172.21.15.44:6819/2220268493","heartbeat_back_addr":"172.21.15.44:6823/2220268493","heartbeat_front_addr":"172.21.15.44:6821/2220268493","state":["exists","up"]},{"osd":1,"uuid":"70a6370a-7c0a-4d64-a0e4-b2e726b4f291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6809","nonce":1278678977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6811","nonce":1278678977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6815","nonce":1278678977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6813","nonce":1278678977}]},"public_addr":"172.21.15.44:6809/1278678977","cluster_addr":"172.21.15.44:6811/1278678977","heartbeat_back_addr":"172.21.15.44:6815/1278678977","heartbeat_front_addr":"172.21.15.44:6813/1278678977","state":["exists","up"]},{"osd":2,"uuid":"f2e51a62-ca91-470d-bd5a-5710c61deee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6801","nonce":2544087903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6803","nonce":2544087903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6807","nonce":2544087903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6805","nonce":2544087903}]},"public_addr":"172.21.15.44:6801/2544087903","cluster_addr":"172.21.15.44:6803/2544087903","heartbeat_back_addr":"172.21.15.44:6807/2544087903","heartbeat_front_addr":"172.21.15.44:6805/2544087903","state":["exists","up"]},{"osd":3,"uuid":"bc514203-9af9-4f1f-b18e-765bdd5de6e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6825","nonce":1692928109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6827","nonce":1692928109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6831","nonce":1692928109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6829","nonce":1692928109}]},"public_addr":"172.21.15.44:6825/1692928109","cluster_addr":"172.21.15.44:6827/1692928109","heartbeat_back_addr":"172.21.15.44:6831/1692928109","heartbeat_front_addr":"172.21.15.44:6829/1692928109","state":["exists","up"]},{"osd":4,"uuid":"357c439a-6e08-4c19-ac18-b68e6c890497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6820","nonce":3802577427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6824","nonce":3802577427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6831","nonce":3802577427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6828","nonce":3802577427}]},"public_addr":"172.21.15.70:6820/3802577427","cluster_addr":"172.21.15.70:6824/3802577427","heartbeat_back_addr":"172.21.15.70:6831/3802577427","heartbeat_front_addr":"172.21.15.70:6828/3802577427","state":["exists","up"]},{"osd":5,"uuid":"952fb88f-d56e-4305-bfa8-d32674332dfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6801","nonce":664757345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6803","nonce":664757345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6807","nonce":664757345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6805","nonce":664757345}]},"public_addr":"172.21.15.70:6801/664757345","cluster_addr":"172.21.15.70:6803/664757345","heartbeat_back_addr":"172.21.15.70:6807/664757345","heartbeat_front_addr":"172.21.15.70:6805/664757345","state":["exists","up"]},{"osd":6,"uuid":"74399b3d-fac2-45ab-9172-2cd95b02450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6817","nonce":1623253482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6821","nonce":1623253482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6827","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6829","nonce":1623253482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6825","nonce":1623253482}]},"public_addr":"172.21.15.70:6817/1623253482","cluster_addr":"172.21.15.70:6821/1623253482","heartbeat_back_addr":"172.21.15.70:6829/1623253482","heartbeat_front_addr":"172.21.15.70:6825/1623253482","state":["exists","up"]},{"osd":7,"uuid":"8f35508d-6fcc-43ff-8ee7-0498a7ab3b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6809","nonce":1551180300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6811","nonce":1551180300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6815","nonce":1551180300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6813","nonce":1551180300}]},"public_addr":"172.21.15.70:6809/1551180300","cluster_addr":"172.21.15.70:6811/1551180300","heartbeat_back_addr":"172.21.15.70:6815/1551180300","heartbeat_front_addr":"172.21.15.70:6813/1551180300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.612502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.653756+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.521169+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.649955+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.640231+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.686638+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.642199+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.700360+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T06:32:09.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.175+0000 7fdd97a77640 1 -- 172.21.15.44:0/1095260379 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fdd7406c8f0 msgr2=0x7fdd7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:09.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.175+0000 7fdd97a77640 1 --2- 172.21.15.44:0/1095260379 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fdd7406c8f0 0x7fdd7406edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdd80001050 tx=0x7fdd80007040 comp rx=0 tx=0).stop 2024-04-22T06:32:09.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.175+0000 7fdd97a77640 1 -- 172.21.15.44:0/1095260379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd7c0051c0 msgr2=0x7fdd900cf830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:09.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.175+0000 7fdd97a77640 1 --2- 172.21.15.44:0/1095260379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd7c0051c0 0x7fdd900cf830 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c011940 tx=0x7fdd8c011e00 comp rx=0 tx=0).stop 2024-04-22T06:32:09.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.175+0000 7fdd97a77640 1 -- 172.21.15.44:0/1095260379 shutdown_connections 2024-04-22T06:32:09.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.175+0000 7fdd97a77640 1 --2- 172.21.15.44:0/1095260379 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fdd7406c8f0 0x7fdd7406edb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:09.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.175+0000 7fdd97a77640 1 --2- 172.21.15.44:0/1095260379 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdd90157000 0x7fdd90148740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:09.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.175+0000 7fdd97a77640 1 --2- 172.21.15.44:0/1095260379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd900d0fa0 0x7fdd90156ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:09.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.175+0000 7fdd97a77640 1 --2- 172.21.15.44:0/1095260379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd7c0051c0 0x7fdd900cf830 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:09.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.175+0000 7fdd97a77640 1 -- 172.21.15.44:0/1095260379 >> 172.21.15.44:0/1095260379 conn(0x7fdd90072fb0 msgr2=0x7fdd900c9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:09.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.175+0000 7fdd97a77640 1 -- 172.21.15.44:0/1095260379 shutdown_connections 2024-04-22T06:32:09.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.175+0000 7fdd97a77640 1 -- 172.21.15.44:0/1095260379 wait complete. 2024-04-22T06:32:09.185 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T06:30:54.560256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T06:31:04.612387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T06:31:05.312124+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-22T06:31:09.025550+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-22T06:32:09.185 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T06:32:09.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.290+0000 7f330b1bd640 1 Processor -- start 2024-04-22T06:32:09.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.290+0000 7f330b1bd640 1 -- start start 2024-04-22T06:32:09.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.291+0000 7f330b1bd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33040c3330 0x7f330406b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:09.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.291+0000 7f330b1bd640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3304155890 con 0x7f33041554b0 2024-04-22T06:32:09.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.291+0000 7f330b1bd640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33040c7980 con 0x7f33040c8430 2024-04-22T06:32:09.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.291+0000 7f330b1bd640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f33040c7c20 con 0x7f33040c3330 2024-04-22T06:32:09.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.291+0000 7f3308f59640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f33041554b0 0x7f330415e660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40184/0 (socket says 172.21.15.44:40184) 2024-04-22T06:32:09.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.291+0000 7f3308f59640 1 -- 172.21.15.44:0/1463093860 learned_addr learned my addr 172.21.15.44:0/1463093860 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:32:09.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.291+0000 7f3303fff640 1 --2- 172.21.15.44:0/1463093860 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33040c3330 0x7f330406b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:09.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.291+0000 7f33037fe640 1 -- 172.21.15.44:0/1463093860 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2688715905 0 0) 0x7f3304155890 con 0x7f33041554b0 2024-04-22T06:32:09.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.292+0000 7f33037fe640 1 -- 172.21.15.44:0/1463093860 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32ec003520 con 0x7f33041554b0 2024-04-22T06:32:09.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.292+0000 7f33037fe640 1 -- 172.21.15.44:0/1463093860 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2047346767 0 0) 0x7f32ec003520 con 0x7f33041554b0 2024-04-22T06:32:09.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.292+0000 7f33037fe640 1 -- 172.21.15.44:0/1463093860 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33040c3330 msgr2=0x7f330406b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:09.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.292+0000 7f33037fe640 1 --2- 172.21.15.44:0/1463093860 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33040c3330 0x7f330406b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:09.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.292+0000 7f33037fe640 1 -- 172.21.15.44:0/1463093860 >> v1:172.21.15.70:6789/0 conn(0x7f33040c8430 legacy=0x7f33041659b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:09.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.292+0000 7f33037fe640 1 -- 172.21.15.44:0/1463093860 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33040c7f00 con 0x7f33041554b0 2024-04-22T06:32:09.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.292+0000 7f33037fe640 1 -- 172.21.15.44:0/1463093860 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1237339257 0 0) 0x7f32fc0019f0 con 0x7f33041554b0 2024-04-22T06:32:09.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.292+0000 7f33037fe640 1 -- 172.21.15.44:0/1463093860 >> v1:172.21.15.44:6789/0 conn(0x7f33041554b0 legacy=0x7f330415e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:09.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.292+0000 7f33037fe640 1 --2- 172.21.15.44:0/1463093860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32ec0039e0 0x7f32ec003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:09.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.292+0000 7f33037fe640 1 --2- 172.21.15.44:0/1463093860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32ec004390 0x7f32ec004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:09.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.292+0000 7f33037fe640 1 --2- 172.21.15.44:0/1463093860 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33040c3330 0x7f32ec0050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:09.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.292+0000 7f33037fe640 1 -- 172.21.15.44:0/1463093860 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f33040c7c20 con 0x7f32ec004390 2024-04-22T06:32:09.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.292+0000 7f33037fe640 1 -- 172.21.15.44:0/1463093860 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f32ec00b9c0 con 0x7f32ec0039e0 2024-04-22T06:32:09.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.292+0000 7f33037fe640 1 -- 172.21.15.44:0/1463093860 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f32ec00bbc0 con 0x7f33040c3330 2024-04-22T06:32:09.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.292+0000 7f3303fff640 1 --2- 172.21.15.44:0/1463093860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32ec004390 0x7f32ec004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:09.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.292+0000 7f3308f59640 1 --2- 172.21.15.44:0/1463093860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32ec0039e0 0x7f32ec003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:09.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.292+0000 7f330975a640 1 --2- 172.21.15.44:0/1463093860 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33040c3330 0x7f32ec0050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:09.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.292+0000 7f3303fff640 1 -- 172.21.15.44:0/1463093860 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33040c3330 msgr2=0x7f32ec0050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:09.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.292+0000 7f3303fff640 1 --2- 172.21.15.44:0/1463093860 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33040c3330 0x7f32ec0050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:09.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.292+0000 7f3303fff640 1 -- 172.21.15.44:0/1463093860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32ec0039e0 msgr2=0x7f32ec003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:09.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.292+0000 7f3303fff640 1 --2- 172.21.15.44:0/1463093860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32ec0039e0 0x7f32ec003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:09.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.292+0000 7f3303fff640 1 -- 172.21.15.44:0/1463093860 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32ec003520 con 0x7f32ec004390 2024-04-22T06:32:09.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.293+0000 7f3303fff640 1 --2- 172.21.15.44:0/1463093860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32ec004390 0x7f32ec004860 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f32f8009ac0 tx=0x7f32ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f278feac054521 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:09.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.293+0000 7f33037fe640 1 -- 172.21.15.44:0/1463093860 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f8039070 con 0x7f32ec004390 2024-04-22T06:32:09.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.293+0000 7f33037fe640 1 -- 172.21.15.44:0/1463093860 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32f802c6b0 con 0x7f32ec004390 2024-04-22T06:32:09.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.293+0000 7f33037fe640 1 -- 172.21.15.44:0/1463093860 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f8009690 con 0x7f32ec004390 2024-04-22T06:32:09.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.294+0000 7f330b1bd640 1 -- 172.21.15.44:0/1463093860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32ec004390 msgr2=0x7f32ec004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:09.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.294+0000 7f330b1bd640 1 --2- 172.21.15.44:0/1463093860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32ec004390 0x7f32ec004860 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f32f8009ac0 tx=0x7f32ec005830 comp rx=0 tx=0).stop 2024-04-22T06:32:09.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.294+0000 7f330975a640 1 -- 172.21.15.44:0/1463093860 reap_dead start 2024-04-22T06:32:09.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.294+0000 7f330b1bd640 1 -- 172.21.15.44:0/1463093860 shutdown_connections 2024-04-22T06:32:09.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.294+0000 7f330b1bd640 1 -- 172.21.15.44:0/1463093860 >> 172.21.15.44:0/1463093860 conn(0x7f33040c9c60 msgr2=0x7f33040ca070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:09.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.294+0000 7f330b1bd640 1 -- 172.21.15.44:0/1463093860 shutdown_connections 2024-04-22T06:32:09.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.294+0000 7f330b1bd640 1 -- 172.21.15.44:0/1463093860 wait complete. 2024-04-22T06:32:09.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.295+0000 7f330b1bd640 1 Processor -- start 2024-04-22T06:32:09.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.295+0000 7f330b1bd640 1 -- start start 2024-04-22T06:32:09.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.296+0000 7f330b1bd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32ec004390 0x7f33040c7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:09.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.296+0000 7f330b1bd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33040afd70 0x7f33041572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:09.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.296+0000 7f330b1bd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33041f1000 0x7f33041f1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:09.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.296+0000 7f330b1bd640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f330415ba00 con 0x7f33041f1000 2024-04-22T06:32:09.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.296+0000 7f330b1bd640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f330415b720 con 0x7f33040afd70 2024-04-22T06:32:09.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.296+0000 7f330b1bd640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f330415b0f0 con 0x7f32ec004390 2024-04-22T06:32:09.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.296+0000 7f330975a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33041f1000 0x7f33041f1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:09.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.296+0000 7f330975a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33041f1000 0x7f33041f1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52574/0 (socket says 172.21.15.44:52574) 2024-04-22T06:32:09.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.296+0000 7f330975a640 1 -- 172.21.15.44:0/3573424925 learned_addr learned my addr 172.21.15.44:0/3573424925 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:09.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.296+0000 7f3303fff640 1 --2- 172.21.15.44:0/3573424925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33040afd70 0x7f33041572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:09.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.296+0000 7f3308f59640 1 --2- 172.21.15.44:0/3573424925 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32ec004390 0x7f33040c7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:09.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.296+0000 7f330975a640 1 -- 172.21.15.44:0/3573424925 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32ec004390 msgr2=0x7f33040c7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:09.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.296+0000 7f330975a640 1 --2- 172.21.15.44:0/3573424925 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32ec004390 0x7f33040c7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:09.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.296+0000 7f330975a640 1 -- 172.21.15.44:0/3573424925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33040afd70 msgr2=0x7f33041572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:09.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.296+0000 7f330975a640 1 --2- 172.21.15.44:0/3573424925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33040afd70 0x7f33041572e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:09.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.296+0000 7f330975a640 1 -- 172.21.15.44:0/3573424925 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33040b1bd0 con 0x7f33041f1000 2024-04-22T06:32:09.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.296+0000 7f3308f59640 1 --2- 172.21.15.44:0/3573424925 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32ec004390 0x7f33040c7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:32:09.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.297+0000 7f330975a640 1 --2- 172.21.15.44:0/3573424925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33041f1000 0x7f33041f1480 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3304071dc0 tx=0x7f32f400fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:09.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.297+0000 7f33017fa640 1 -- 172.21.15.44:0/3573424925 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f4019070 con 0x7f33041f1000 2024-04-22T06:32:09.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.297+0000 7f33017fa640 1 -- 172.21.15.44:0/3573424925 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32f4005030 con 0x7f33041f1000 2024-04-22T06:32:09.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.297+0000 7f330b1bd640 1 -- 172.21.15.44:0/3573424925 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33040b2470 con 0x7f33041f1000 2024-04-22T06:32:09.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.297+0000 7f33017fa640 1 -- 172.21.15.44:0/3573424925 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f4005400 con 0x7f33041f1000 2024-04-22T06:32:09.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.297+0000 7f330b1bd640 1 -- 172.21.15.44:0/3573424925 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33040b2a30 con 0x7f33041f1000 2024-04-22T06:32:09.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.298+0000 7f330b1bd640 1 -- 172.21.15.44:0/3573424925 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f330406a070 con 0x7f33041f1000 2024-04-22T06:32:09.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.298+0000 7f33017fa640 1 -- 172.21.15.44:0/3573424925 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f32f401c070 con 0x7f33041f1000 2024-04-22T06:32:09.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.299+0000 7f33017fa640 1 --2- 172.21.15.44:0/3573424925 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f32f006c8f0 0x7f32f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:09.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.299+0000 7f33017fa640 1 -- 172.21.15.44:0/3573424925 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f32f408af10 con 0x7f33041f1000 2024-04-22T06:32:09.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.299+0000 7f3308f59640 1 --2- 172.21.15.44:0/3573424925 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f32f006c8f0 0x7f32f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:09.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.299+0000 7f3308f59640 1 --2- 172.21.15.44:0/3573424925 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f32f006c8f0 0x7f32f006edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f32fc000f80 tx=0x7f32fc006040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:09.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.303+0000 7f33017fa640 1 -- 172.21.15.44:0/3573424925 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f330406a070 con 0x7f33041f1000 2024-04-22T06:32:09.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.485+0000 7f330b1bd640 1 -- 172.21.15.44:0/3573424925 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3304068cc0 con 0x7f33041f1000 2024-04-22T06:32:09.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.486+0000 7f33017fa640 1 -- 172.21.15.44:0/3573424925 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f32f4058850 con 0x7f33041f1000 2024-04-22T06:32:09.484 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-04-22T06:32:09.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.487+0000 7f330b1bd640 1 -- 172.21.15.44:0/3573424925 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f32f006c8f0 msgr2=0x7f32f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:09.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.487+0000 7f330b1bd640 1 --2- 172.21.15.44:0/3573424925 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f32f006c8f0 0x7f32f006edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f32fc000f80 tx=0x7f32fc006040 comp rx=0 tx=0).stop 2024-04-22T06:32:09.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.487+0000 7f330b1bd640 1 -- 172.21.15.44:0/3573424925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33041f1000 msgr2=0x7f33041f1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:09.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.487+0000 7f330b1bd640 1 --2- 172.21.15.44:0/3573424925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33041f1000 0x7f33041f1480 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3304071dc0 tx=0x7f32f400fc00 comp rx=0 tx=0).stop 2024-04-22T06:32:09.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.487+0000 7f330b1bd640 1 -- 172.21.15.44:0/3573424925 shutdown_connections 2024-04-22T06:32:09.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.487+0000 7f330b1bd640 1 --2- 172.21.15.44:0/3573424925 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f32f006c8f0 0x7f32f006edb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:09.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.487+0000 7f330b1bd640 1 --2- 172.21.15.44:0/3573424925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33041f1000 0x7f33041f1480 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:09.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.487+0000 7f330b1bd640 1 --2- 172.21.15.44:0/3573424925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33040afd70 0x7f33041572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:09.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.487+0000 7f330b1bd640 1 --2- 172.21.15.44:0/3573424925 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32ec004390 0x7f33040c7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:09.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.487+0000 7f330b1bd640 1 -- 172.21.15.44:0/3573424925 >> 172.21.15.44:0/3573424925 conn(0x7f33040c9c60 msgr2=0x7f330406fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:09.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.488+0000 7f330b1bd640 1 -- 172.21.15.44:0/3573424925 shutdown_connections 2024-04-22T06:32:09.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.488+0000 7f330b1bd640 1 -- 172.21.15.44:0/3573424925 wait complete. 2024-04-22T06:32:09.497 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T06:32:09.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.602+0000 7f21a0df4640 1 Processor -- start 2024-04-22T06:32:09.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.602+0000 7f21a0df4640 1 -- start start 2024-04-22T06:32:09.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.603+0000 7f21a0df4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f219c1551e0 0x7f219c15bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:09.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.603+0000 7f21a0df4640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f219c0713b0 0x7f219c06bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:09.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.603+0000 7f21a0df4640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f219c06d4e0 con 0x7f219c06b930 2024-04-22T06:32:09.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.603+0000 7f21a0df4640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f219c070810 con 0x7f219c0713b0 2024-04-22T06:32:09.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.603+0000 7f21a0df4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f219c073380 con 0x7f219c1551e0 2024-04-22T06:32:09.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.603+0000 7f219ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f219c0713b0 0x7f219c06bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:09.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.603+0000 7f2199d9b640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f219c06b930 0x7f219c070ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40188/0 (socket says 172.21.15.44:40188) 2024-04-22T06:32:09.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.603+0000 7f2199d9b640 1 -- 172.21.15.44:0/3061892224 learned_addr learned my addr 172.21.15.44:0/3061892224 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:32:09.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.604+0000 7f219a59c640 1 --2- 172.21.15.44:0/3061892224 >> v2:172.21.15.70:3300/0 conn(0x7f219c1551e0 0x7f219c15bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:09.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.604+0000 7f219ad9d640 1 -- 172.21.15.44:0/3061892224 >> v2:172.21.15.70:3300/0 conn(0x7f219c1551e0 msgr2=0x7f219c15bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:09.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.604+0000 7f219ad9d640 1 --2- 172.21.15.44:0/3061892224 >> v2:172.21.15.70:3300/0 conn(0x7f219c1551e0 0x7f219c15bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:09.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.604+0000 7f219ad9d640 1 -- 172.21.15.44:0/3061892224 >> v1:172.21.15.44:6789/0 conn(0x7f219c06b930 legacy=0x7f219c070ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:09.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.604+0000 7f219ad9d640 1 -- 172.21.15.44:0/3061892224 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f219c073660 con 0x7f219c0713b0 2024-04-22T06:32:09.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.604+0000 7f219ad9d640 1 --2- 172.21.15.44:0/3061892224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f219c0713b0 0x7f219c06bda0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f219c068930 tx=0x7f218c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b177690967538d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:09.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.605+0000 7f219959a640 1 -- 172.21.15.44:0/3061892224 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218c039070 con 0x7f219c0713b0 2024-04-22T06:32:09.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.605+0000 7f219959a640 1 -- 172.21.15.44:0/3061892224 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f218c02cc80 con 0x7f219c0713b0 2024-04-22T06:32:09.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.605+0000 7f219959a640 1 -- 172.21.15.44:0/3061892224 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218c034660 con 0x7f219c0713b0 2024-04-22T06:32:09.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.605+0000 7f21a0df4640 1 -- 172.21.15.44:0/3061892224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f219c0713b0 msgr2=0x7f219c06bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:09.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.605+0000 7f21a0df4640 1 --2- 172.21.15.44:0/3061892224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f219c0713b0 0x7f219c06bda0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f219c068930 tx=0x7f218c02bed0 comp rx=0 tx=0).stop 2024-04-22T06:32:09.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.605+0000 7f21a0df4640 1 -- 172.21.15.44:0/3061892224 shutdown_connections 2024-04-22T06:32:09.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.605+0000 7f21a0df4640 1 --2- 172.21.15.44:0/3061892224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f219c0713b0 0x7f219c06bda0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:09.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.605+0000 7f21a0df4640 1 --2- 172.21.15.44:0/3061892224 >> v2:172.21.15.70:3300/0 conn(0x7f219c1551e0 0x7f219c15bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:09.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.605+0000 7f21a0df4640 1 -- 172.21.15.44:0/3061892224 >> 172.21.15.44:0/3061892224 conn(0x7f219c0ce700 msgr2=0x7f219c0ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:09.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.605+0000 7f21a0df4640 1 -- 172.21.15.44:0/3061892224 shutdown_connections 2024-04-22T06:32:09.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.605+0000 7f21a0df4640 1 -- 172.21.15.44:0/3061892224 wait complete. 2024-04-22T06:32:09.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.607+0000 7f21a0df4640 1 Processor -- start 2024-04-22T06:32:09.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.607+0000 7f21a0df4640 1 -- start start 2024-04-22T06:32:09.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.607+0000 7f21a0df4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f219c06b930 0x7f219c142950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:09.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.607+0000 7f21a0df4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f219c0713b0 0x7f219c1f46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:09.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.607+0000 7f21a0df4640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f219c1551e0 0x7f219c153450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:09.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.607+0000 7f21a0df4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f219c0cf420 con 0x7f219c0713b0 2024-04-22T06:32:09.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.607+0000 7f21a0df4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f219c073c20 con 0x7f219c06b930 2024-04-22T06:32:09.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.607+0000 7f21a0df4640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f219c1555c0 con 0x7f219c1551e0 2024-04-22T06:32:09.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.607+0000 7f2199d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f219c0713b0 0x7f219c1f46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:09.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.607+0000 7f2199d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f219c0713b0 0x7f219c1f46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52598/0 (socket says 172.21.15.44:52598) 2024-04-22T06:32:09.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.607+0000 7f2199d9b640 1 -- 172.21.15.44:0/1854378638 learned_addr learned my addr 172.21.15.44:0/1854378638 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:09.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.608+0000 7f219a59c640 1 --2- 172.21.15.44:0/1854378638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f219c06b930 0x7f219c142950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:09.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.608+0000 7f219ad9d640 1 --2- 172.21.15.44:0/1854378638 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f219c1551e0 0x7f219c153450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:09.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.608+0000 7f2199d9b640 1 -- 172.21.15.44:0/1854378638 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f219c1551e0 msgr2=0x7f219c153450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:09.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.608+0000 7f2199d9b640 1 --2- 172.21.15.44:0/1854378638 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f219c1551e0 0x7f219c153450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:09.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.608+0000 7f2199d9b640 1 -- 172.21.15.44:0/1854378638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f219c06b930 msgr2=0x7f219c142950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:09.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.608+0000 7f2199d9b640 1 --2- 172.21.15.44:0/1854378638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f219c06b930 0x7f219c142950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:09.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.608+0000 7f2199d9b640 1 -- 172.21.15.44:0/1854378638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f219c1591b0 con 0x7f219c0713b0 2024-04-22T06:32:09.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.608+0000 7f2199d9b640 1 --2- 172.21.15.44:0/1854378638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f219c0713b0 0x7f219c1f46a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f218400c4b0 tx=0x7f218400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:09.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.608+0000 7f2192ffd640 1 -- 172.21.15.44:0/1854378638 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2184017070 con 0x7f219c0713b0 2024-04-22T06:32:09.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.608+0000 7f2192ffd640 1 -- 172.21.15.44:0/1854378638 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2184005ce0 con 0x7f219c0713b0 2024-04-22T06:32:09.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.608+0000 7f21a0df4640 1 -- 172.21.15.44:0/1854378638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f219c155ca0 con 0x7f219c0713b0 2024-04-22T06:32:09.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.608+0000 7f2192ffd640 1 -- 172.21.15.44:0/1854378638 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2184007270 con 0x7f219c0713b0 2024-04-22T06:32:09.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.608+0000 7f21a0df4640 1 -- 172.21.15.44:0/1854378638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f219c156ad0 con 0x7f219c0713b0 2024-04-22T06:32:09.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.609+0000 7f21a0df4640 1 -- 172.21.15.44:0/1854378638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f219c1561c0 con 0x7f219c0713b0 2024-04-22T06:32:09.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.610+0000 7f2192ffd640 1 -- 172.21.15.44:0/1854378638 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f218401a070 con 0x7f219c0713b0 2024-04-22T06:32:09.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.610+0000 7f2192ffd640 1 --2- 172.21.15.44:0/1854378638 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f217c06c920 0x7f217c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:09.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.610+0000 7f2192ffd640 1 -- 172.21.15.44:0/1854378638 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f218408b180 con 0x7f219c0713b0 2024-04-22T06:32:09.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.610+0000 7f219a59c640 1 --2- 172.21.15.44:0/1854378638 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f217c06c920 0x7f217c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:09.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.611+0000 7f219a59c640 1 --2- 172.21.15.44:0/1854378638 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f217c06c920 0x7f217c06ede0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2194006fd0 tx=0x7f2194008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:09.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.615+0000 7f2192ffd640 1 -- 172.21.15.44:0/1854378638 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f219c1561c0 con 0x7f219c0713b0 2024-04-22T06:32:09.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.800+0000 7f21a0df4640 1 -- 172.21.15.44:0/1854378638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f219c069310 con 0x7f219c0713b0 2024-04-22T06:32:09.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.800+0000 7f2192ffd640 1 -- 172.21.15.44:0/1854378638 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f2184058b60 con 0x7f219c0713b0 2024-04-22T06:32:09.799 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-04-22T06:32:09.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.802+0000 7f21a0df4640 1 -- 172.21.15.44:0/1854378638 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f217c06c920 msgr2=0x7f217c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:09.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.802+0000 7f21a0df4640 1 --2- 172.21.15.44:0/1854378638 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f217c06c920 0x7f217c06ede0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2194006fd0 tx=0x7f2194008040 comp rx=0 tx=0).stop 2024-04-22T06:32:09.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.802+0000 7f21a0df4640 1 -- 172.21.15.44:0/1854378638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f219c0713b0 msgr2=0x7f219c1f46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:09.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.802+0000 7f21a0df4640 1 --2- 172.21.15.44:0/1854378638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f219c0713b0 0x7f219c1f46a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f218400c4b0 tx=0x7f218400c970 comp rx=0 tx=0).stop 2024-04-22T06:32:09.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.802+0000 7f21a0df4640 1 -- 172.21.15.44:0/1854378638 shutdown_connections 2024-04-22T06:32:09.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.802+0000 7f21a0df4640 1 --2- 172.21.15.44:0/1854378638 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f217c06c920 0x7f217c06ede0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:09.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.802+0000 7f21a0df4640 1 --2- 172.21.15.44:0/1854378638 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f219c1551e0 0x7f219c153450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:09.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.802+0000 7f21a0df4640 1 --2- 172.21.15.44:0/1854378638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f219c0713b0 0x7f219c1f46a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:09.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.802+0000 7f21a0df4640 1 --2- 172.21.15.44:0/1854378638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f219c06b930 0x7f219c142950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:09.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.802+0000 7f21a0df4640 1 -- 172.21.15.44:0/1854378638 >> 172.21.15.44:0/1854378638 conn(0x7f219c0ce700 msgr2=0x7f219c14b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:09.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.802+0000 7f21a0df4640 1 -- 172.21.15.44:0/1854378638 shutdown_connections 2024-04-22T06:32:09.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.802+0000 7f21a0df4640 1 -- 172.21.15.44:0/1854378638 wait complete. 2024-04-22T06:32:09.812 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T06:32:09.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.917+0000 7f1382952640 1 Processor -- start 2024-04-22T06:32:09.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.917+0000 7f1382952640 1 -- start start 2024-04-22T06:32:09.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.918+0000 7f1382952640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f137c0c9190 0x7f137c0c3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:09.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.918+0000 7f1382952640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f137c0c9570 0x7f137c0c45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:09.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.918+0000 7f1382952640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f137c0c6f10 con 0x7f137c0c8970 2024-04-22T06:32:09.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.918+0000 7f1382952640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f137c158300 con 0x7f137c0c9190 2024-04-22T06:32:09.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.918+0000 7f1382952640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f137c0c71b0 con 0x7f137c0c9570 2024-04-22T06:32:09.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.918+0000 7f137bfff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f137c0c9190 0x7f137c0c3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:09.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.918+0000 7f137bfff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f137c0c9190 0x7f137c0c3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52614/0 (socket says 172.21.15.44:52614) 2024-04-22T06:32:09.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.918+0000 7f1380eef640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f137c0c8970 0x7f137c151550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40194/0 (socket says 172.21.15.44:40194) 2024-04-22T06:32:09.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.918+0000 7f137bfff640 1 -- 172.21.15.44:0/3217813284 learned_addr learned my addr 172.21.15.44:0/3217813284 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:09.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.918+0000 7f137b7fe640 1 --2- 172.21.15.44:0/3217813284 >> v2:172.21.15.70:3300/0 conn(0x7f137c0c9570 0x7f137c0c45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:09.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.918+0000 7f137bfff640 1 -- 172.21.15.44:0/3217813284 >> v2:172.21.15.70:3300/0 conn(0x7f137c0c9570 msgr2=0x7f137c0c45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:09.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.918+0000 7f137bfff640 1 --2- 172.21.15.44:0/3217813284 >> v2:172.21.15.70:3300/0 conn(0x7f137c0c9570 0x7f137c0c45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:09.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.918+0000 7f137bfff640 1 -- 172.21.15.44:0/3217813284 >> v1:172.21.15.44:6789/0 conn(0x7f137c0c8970 legacy=0x7f137c151550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:09.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.918+0000 7f137bfff640 1 -- 172.21.15.44:0/3217813284 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f137c0c7490 con 0x7f137c0c9190 2024-04-22T06:32:09.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.918+0000 7f137affd640 1 -- 172.21.15.44:0/3217813284 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2310464013 0 0) 0x7f137c0c6f10 con 0x7f137c0c8970 2024-04-22T06:32:09.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.919+0000 7f137bfff640 1 --2- 172.21.15.44:0/3217813284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0c9190 0x7f137c0c3910 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f137c06dc60 tx=0x7f136c02ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3cb33e9b4512961 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:09.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.919+0000 7f137affd640 1 -- 172.21.15.44:0/3217813284 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136c02f850 con 0x7f137c0c9190 2024-04-22T06:32:09.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.919+0000 7f137affd640 1 -- 172.21.15.44:0/3217813284 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f136c02f9f0 con 0x7f137c0c9190 2024-04-22T06:32:09.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.919+0000 7f137affd640 1 -- 172.21.15.44:0/3217813284 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136c02fdc0 con 0x7f137c0c9190 2024-04-22T06:32:09.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.920+0000 7f1382952640 1 -- 172.21.15.44:0/3217813284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0c9190 msgr2=0x7f137c0c3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:09.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.920+0000 7f1382952640 1 --2- 172.21.15.44:0/3217813284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0c9190 0x7f137c0c3910 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f137c06dc60 tx=0x7f136c02ba30 comp rx=0 tx=0).stop 2024-04-22T06:32:09.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.920+0000 7f1382952640 1 -- 172.21.15.44:0/3217813284 shutdown_connections 2024-04-22T06:32:09.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.920+0000 7f1382952640 1 --2- 172.21.15.44:0/3217813284 >> v2:172.21.15.70:3300/0 conn(0x7f137c0c9570 0x7f137c0c45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:09.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.920+0000 7f1382952640 1 --2- 172.21.15.44:0/3217813284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0c9190 0x7f137c0c3910 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:09.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.920+0000 7f1382952640 1 -- 172.21.15.44:0/3217813284 >> 172.21.15.44:0/3217813284 conn(0x7f137c06d530 msgr2=0x7f137c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:09.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.920+0000 7f1382952640 1 -- 172.21.15.44:0/3217813284 shutdown_connections 2024-04-22T06:32:09.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.920+0000 7f1382952640 1 -- 172.21.15.44:0/3217813284 wait complete. 2024-04-22T06:32:09.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.921+0000 7f1382952640 1 Processor -- start 2024-04-22T06:32:09.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.921+0000 7f1382952640 1 -- start start 2024-04-22T06:32:09.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.922+0000 7f1382952640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f137c0c8970 0x7f137c154a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:09.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.922+0000 7f1382952640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0c9190 0x7f137c0cae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:09.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.922+0000 7f1382952640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f137c0c9570 0x7f137c0cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:09.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.922+0000 7f1382952640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f137c0c4b30 con 0x7f137c0c9190 2024-04-22T06:32:09.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.922+0000 7f1382952640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f137c1585e0 con 0x7f137c0c8970 2024-04-22T06:32:09.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.922+0000 7f1382952640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f137c0c7c90 con 0x7f137c0c9570 2024-04-22T06:32:09.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.922+0000 7f137b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0c9190 0x7f137c0cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:09.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.922+0000 7f137b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0c9190 0x7f137c0cae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52620/0 (socket says 172.21.15.44:52620) 2024-04-22T06:32:09.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.922+0000 7f137b7fe640 1 -- 172.21.15.44:0/3692449850 learned_addr learned my addr 172.21.15.44:0/3692449850 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:09.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.922+0000 7f137bfff640 1 --2- 172.21.15.44:0/3692449850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f137c0c8970 0x7f137c154a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:09.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.922+0000 7f1380eef640 1 --2- 172.21.15.44:0/3692449850 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f137c0c9570 0x7f137c0cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:09.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.922+0000 7f137b7fe640 1 -- 172.21.15.44:0/3692449850 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f137c0c9570 msgr2=0x7f137c0cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:09.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.922+0000 7f137b7fe640 1 --2- 172.21.15.44:0/3692449850 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f137c0c9570 0x7f137c0cb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:09.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.922+0000 7f137b7fe640 1 -- 172.21.15.44:0/3692449850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f137c0c8970 msgr2=0x7f137c154a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:09.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.922+0000 7f137b7fe640 1 --2- 172.21.15.44:0/3692449850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f137c0c8970 0x7f137c154a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:09.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.922+0000 7f137b7fe640 1 -- 172.21.15.44:0/3692449850 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f137c1ef3f0 con 0x7f137c0c9190 2024-04-22T06:32:09.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.922+0000 7f137bfff640 1 --2- 172.21.15.44:0/3692449850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f137c0c8970 0x7f137c154a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:32:09.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.922+0000 7f1380eef640 1 --2- 172.21.15.44:0/3692449850 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f137c0c9570 0x7f137c0cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:32:09.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.922+0000 7f137b7fe640 1 --2- 172.21.15.44:0/3692449850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0c9190 0x7f137c0cae60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f137000d620 tx=0x7f137000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:09.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.922+0000 7f1378ff9640 1 -- 172.21.15.44:0/3692449850 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137000b620 con 0x7f137c0c9190 2024-04-22T06:32:09.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.922+0000 7f1378ff9640 1 -- 172.21.15.44:0/3692449850 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1370019070 con 0x7f137c0c9190 2024-04-22T06:32:09.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.923+0000 7f1378ff9640 1 -- 172.21.15.44:0/3692449850 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137000ba00 con 0x7f137c0c9190 2024-04-22T06:32:09.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.923+0000 7f1382952640 1 -- 172.21.15.44:0/3692449850 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f137c1567d0 con 0x7f137c0c9190 2024-04-22T06:32:09.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.923+0000 7f1382952640 1 -- 172.21.15.44:0/3692449850 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f137c156060 con 0x7f137c0c9190 2024-04-22T06:32:09.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.923+0000 7f1382952640 1 -- 172.21.15.44:0/3692449850 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f137c155e50 con 0x7f137c0c9190 2024-04-22T06:32:09.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.924+0000 7f1378ff9640 1 -- 172.21.15.44:0/3692449850 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f137000bba0 con 0x7f137c0c9190 2024-04-22T06:32:09.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.924+0000 7f1378ff9640 1 --2- 172.21.15.44:0/3692449850 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f136806c8f0 0x7f136806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:09.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.924+0000 7f1378ff9640 1 -- 172.21.15.44:0/3692449850 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f137008c2b0 con 0x7f137c0c9190 2024-04-22T06:32:09.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.925+0000 7f137bfff640 1 --2- 172.21.15.44:0/3692449850 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f136806c8f0 0x7f136806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:09.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.925+0000 7f137bfff640 1 --2- 172.21.15.44:0/3692449850 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f136806c8f0 0x7f136806edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f136c036000 tx=0x7f136c039040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:09.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:09.929+0000 7f1378ff9640 1 -- 172.21.15.44:0/3692449850 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f137005a370 con 0x7f137c0c9190 2024-04-22T06:32:10.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.114+0000 7f1382952640 1 -- 172.21.15.44:0/3692449850 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f137c068cc0 con 0x7f137c0c9190 2024-04-22T06:32:10.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.115+0000 7f1378ff9640 1 -- 172.21.15.44:0/3692449850 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f1370060030 con 0x7f137c0c9190 2024-04-22T06:32:10.114 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-04-22T06:32:10.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.116+0000 7f1382952640 1 -- 172.21.15.44:0/3692449850 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f136806c8f0 msgr2=0x7f136806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:10.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.116+0000 7f1382952640 1 --2- 172.21.15.44:0/3692449850 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f136806c8f0 0x7f136806edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f136c036000 tx=0x7f136c039040 comp rx=0 tx=0).stop 2024-04-22T06:32:10.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.116+0000 7f1382952640 1 -- 172.21.15.44:0/3692449850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0c9190 msgr2=0x7f137c0cae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:10.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.116+0000 7f1382952640 1 --2- 172.21.15.44:0/3692449850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0c9190 0x7f137c0cae60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f137000d620 tx=0x7f137000dae0 comp rx=0 tx=0).stop 2024-04-22T06:32:10.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.117+0000 7f1382952640 1 -- 172.21.15.44:0/3692449850 shutdown_connections 2024-04-22T06:32:10.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.117+0000 7f1382952640 1 --2- 172.21.15.44:0/3692449850 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f136806c8f0 0x7f136806edb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:10.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.117+0000 7f1382952640 1 --2- 172.21.15.44:0/3692449850 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f137c0c9570 0x7f137c0cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:10.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.117+0000 7f1382952640 1 --2- 172.21.15.44:0/3692449850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0c9190 0x7f137c0cae60 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:10.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.117+0000 7f1382952640 1 --2- 172.21.15.44:0/3692449850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f137c0c8970 0x7f137c154a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:10.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.117+0000 7f1382952640 1 -- 172.21.15.44:0/3692449850 >> 172.21.15.44:0/3692449850 conn(0x7f137c06d530 msgr2=0x7f137c14fc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:10.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.117+0000 7f1382952640 1 -- 172.21.15.44:0/3692449850 shutdown_connections 2024-04-22T06:32:10.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.117+0000 7f1382952640 1 -- 172.21.15.44:0/3692449850 wait complete. 2024-04-22T06:32:10.126 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-22T06:32:10.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.232+0000 7f9a9b669640 1 Processor -- start 2024-04-22T06:32:10.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.232+0000 7f9a9b669640 1 -- start start 2024-04-22T06:32:10.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.233+0000 7f9a9b669640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9a94074310 0x7f9a94166aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:10.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.233+0000 7f9a9b669640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a940c94b0 con 0x7f9a9406bba0 2024-04-22T06:32:10.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.233+0000 7f9a9b669640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a940c9b00 con 0x7f9a9406bf80 2024-04-22T06:32:10.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.233+0000 7f9a9b669640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f9a940c9de0 con 0x7f9a94074310 2024-04-22T06:32:10.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.233+0000 7f9a99405640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f9a9406bba0 0x7f9a94154f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40200/0 (socket says 172.21.15.44:40200) 2024-04-22T06:32:10.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.233+0000 7f9a99405640 1 -- 172.21.15.44:0/518850916 learned_addr learned my addr 172.21.15.44:0/518850916 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:32:10.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.233+0000 7f9a99c06640 1 --2- 172.21.15.44:0/518850916 >> v2:172.21.15.70:3300/0 conn(0x7f9a94074310 0x7f9a94166aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:10.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.233+0000 7f9a8bfff640 1 -- 172.21.15.44:0/518850916 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 645026061 0 0) 0x7f9a940c94b0 con 0x7f9a9406bba0 2024-04-22T06:32:10.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.233+0000 7f9a8bfff640 1 -- 172.21.15.44:0/518850916 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a80003520 con 0x7f9a9406bba0 2024-04-22T06:32:10.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.233+0000 7f9a99c06640 1 -- 172.21.15.44:0/518850916 >> v1:172.21.15.70:6789/0 conn(0x7f9a9406bf80 legacy=0x7f9a9415e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:10.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.233+0000 7f9a8bfff640 1 -- 172.21.15.44:0/518850916 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3048038697 0 0) 0x7f9a80003520 con 0x7f9a9406bba0 2024-04-22T06:32:10.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.233+0000 7f9a99c06640 1 -- 172.21.15.44:0/518850916 >> v1:172.21.15.44:6789/0 conn(0x7f9a9406bba0 legacy=0x7f9a94154f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:10.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.233+0000 7f9a99c06640 1 -- 172.21.15.44:0/518850916 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a940700f0 con 0x7f9a94074310 2024-04-22T06:32:10.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.234+0000 7f9a99c06640 1 --2- 172.21.15.44:0/518850916 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a94074310 0x7f9a94166aa0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c009d90 tx=0x7f9a8c02bd00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93f25fcecb190223 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:10.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.234+0000 7f9a8bfff640 1 -- 172.21.15.44:0/518850916 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c039070 con 0x7f9a94074310 2024-04-22T06:32:10.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.234+0000 7f9a8bfff640 1 -- 172.21.15.44:0/518850916 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a8c02c830 con 0x7f9a94074310 2024-04-22T06:32:10.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.234+0000 7f9a8bfff640 1 -- 172.21.15.44:0/518850916 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a90001670 con 0x7f9a94074310 2024-04-22T06:32:10.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.235+0000 7f9a9b669640 1 -- 172.21.15.44:0/518850916 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a94074310 msgr2=0x7f9a94166aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:10.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.235+0000 7f9a9b669640 1 --2- 172.21.15.44:0/518850916 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a94074310 0x7f9a94166aa0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c009d90 tx=0x7f9a8c02bd00 comp rx=0 tx=0).stop 2024-04-22T06:32:10.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.235+0000 7f9a9b669640 1 -- 172.21.15.44:0/518850916 shutdown_connections 2024-04-22T06:32:10.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.235+0000 7f9a9b669640 1 --2- 172.21.15.44:0/518850916 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a94074310 0x7f9a94166aa0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:10.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.235+0000 7f9a9b669640 1 -- 172.21.15.44:0/518850916 >> 172.21.15.44:0/518850916 conn(0x7f9a940cd5f0 msgr2=0x7f9a940cda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:10.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.235+0000 7f9a9b669640 1 -- 172.21.15.44:0/518850916 shutdown_connections 2024-04-22T06:32:10.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.235+0000 7f9a9b669640 1 -- 172.21.15.44:0/518850916 wait complete. 2024-04-22T06:32:10.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.236+0000 7f9a9b669640 1 Processor -- start 2024-04-22T06:32:10.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.236+0000 7f9a9b669640 1 -- start start 2024-04-22T06:32:10.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.237+0000 7f9a9b669640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a9406bba0 0x7f9a940c6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:10.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.237+0000 7f9a9b669640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a9406bf80 0x7f9a9415dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:10.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.237+0000 7f9a9b669640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9a94074310 0x7f9a940c48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:10.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.237+0000 7f9a9b669640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9a94155660 con 0x7f9a9406bf80 2024-04-22T06:32:10.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.237+0000 7f9a9b669640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9a940c8f30 con 0x7f9a9406bba0 2024-04-22T06:32:10.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.237+0000 7f9a9b669640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9a941557e0 con 0x7f9a94074310 2024-04-22T06:32:10.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.237+0000 7f9a98c04640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a9406bf80 0x7f9a9415dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:10.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.237+0000 7f9a98c04640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a9406bf80 0x7f9a9415dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52630/0 (socket says 172.21.15.44:52630) 2024-04-22T06:32:10.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.237+0000 7f9a98c04640 1 -- 172.21.15.44:0/1573029488 learned_addr learned my addr 172.21.15.44:0/1573029488 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:10.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.237+0000 7f9a99405640 1 --2- 172.21.15.44:0/1573029488 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a9406bba0 0x7f9a940c6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:10.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.237+0000 7f9a99c06640 1 --2- 172.21.15.44:0/1573029488 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9a94074310 0x7f9a940c48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:10.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.237+0000 7f9a98c04640 1 -- 172.21.15.44:0/1573029488 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9a94074310 msgr2=0x7f9a940c48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:10.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.237+0000 7f9a98c04640 1 --2- 172.21.15.44:0/1573029488 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9a94074310 0x7f9a940c48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:10.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.237+0000 7f9a98c04640 1 -- 172.21.15.44:0/1573029488 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a9406bba0 msgr2=0x7f9a940c6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:10.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.237+0000 7f9a98c04640 1 --2- 172.21.15.44:0/1573029488 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a9406bba0 0x7f9a940c6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:10.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.237+0000 7f9a98c04640 1 -- 172.21.15.44:0/1573029488 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a940c7f50 con 0x7f9a9406bf80 2024-04-22T06:32:10.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.238+0000 7f9a98c04640 1 --2- 172.21.15.44:0/1573029488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a9406bf80 0x7f9a9415dd40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9a8400c780 tx=0x7f9a8400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:10.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.238+0000 7f9a89ffb640 1 -- 172.21.15.44:0/1573029488 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a84017070 con 0x7f9a9406bf80 2024-04-22T06:32:10.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.238+0000 7f9a89ffb640 1 -- 172.21.15.44:0/1573029488 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a84005270 con 0x7f9a9406bf80 2024-04-22T06:32:10.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.238+0000 7f9a9b669640 1 -- 172.21.15.44:0/1573029488 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a941f1f80 con 0x7f9a9406bf80 2024-04-22T06:32:10.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.238+0000 7f9a89ffb640 1 -- 172.21.15.44:0/1573029488 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a840035f0 con 0x7f9a9406bf80 2024-04-22T06:32:10.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.238+0000 7f9a9b669640 1 -- 172.21.15.44:0/1573029488 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a940c6df0 con 0x7f9a9406bf80 2024-04-22T06:32:10.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.239+0000 7f9a9b669640 1 -- 172.21.15.44:0/1573029488 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a94068cc0 con 0x7f9a9406bf80 2024-04-22T06:32:10.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.239+0000 7f9a89ffb640 1 -- 172.21.15.44:0/1573029488 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a8401a070 con 0x7f9a9406bf80 2024-04-22T06:32:10.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.240+0000 7f9a89ffb640 1 --2- 172.21.15.44:0/1573029488 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f9a7806c920 0x7f9a7806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:10.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.240+0000 7f9a89ffb640 1 -- 172.21.15.44:0/1573029488 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f9a8408b380 con 0x7f9a9406bf80 2024-04-22T06:32:10.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.240+0000 7f9a99405640 1 --2- 172.21.15.44:0/1573029488 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f9a7806c920 0x7f9a7806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:10.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.240+0000 7f9a99405640 1 --2- 172.21.15.44:0/1573029488 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f9a7806c920 0x7f9a7806ede0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9a90001eb0 tx=0x7f9a90001e00 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:10.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.244+0000 7f9a89ffb640 1 -- 172.21.15.44:0/1573029488 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9a84058cd0 con 0x7f9a9406bf80 2024-04-22T06:32:10.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.426+0000 7f9a9b669640 1 -- 172.21.15.44:0/1573029488 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f9a9414aa20 con 0x7f9a9406bf80 2024-04-22T06:32:10.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.427+0000 7f9a89ffb640 1 -- 172.21.15.44:0/1573029488 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f9a8405cfd0 con 0x7f9a9406bf80 2024-04-22T06:32:10.426 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-04-22T06:32:10.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.428+0000 7f9a9b669640 1 -- 172.21.15.44:0/1573029488 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f9a7806c920 msgr2=0x7f9a7806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:10.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.428+0000 7f9a9b669640 1 --2- 172.21.15.44:0/1573029488 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f9a7806c920 0x7f9a7806ede0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9a90001eb0 tx=0x7f9a90001e00 comp rx=0 tx=0).stop 2024-04-22T06:32:10.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.429+0000 7f9a9b669640 1 -- 172.21.15.44:0/1573029488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a9406bf80 msgr2=0x7f9a9415dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:10.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.429+0000 7f9a9b669640 1 --2- 172.21.15.44:0/1573029488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a9406bf80 0x7f9a9415dd40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9a8400c780 tx=0x7f9a8400cc40 comp rx=0 tx=0).stop 2024-04-22T06:32:10.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.429+0000 7f9a9b669640 1 -- 172.21.15.44:0/1573029488 shutdown_connections 2024-04-22T06:32:10.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.429+0000 7f9a9b669640 1 --2- 172.21.15.44:0/1573029488 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f9a7806c920 0x7f9a7806ede0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:10.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.429+0000 7f9a9b669640 1 --2- 172.21.15.44:0/1573029488 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9a94074310 0x7f9a940c48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:10.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.429+0000 7f9a9b669640 1 --2- 172.21.15.44:0/1573029488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a9406bf80 0x7f9a9415dd40 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:10.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.429+0000 7f9a9b669640 1 --2- 172.21.15.44:0/1573029488 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a9406bba0 0x7f9a940c6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:10.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.429+0000 7f9a9b669640 1 -- 172.21.15.44:0/1573029488 >> 172.21.15.44:0/1573029488 conn(0x7f9a940cd5f0 msgr2=0x7f9a940cd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:10.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.429+0000 7f9a9b669640 1 -- 172.21.15.44:0/1573029488 shutdown_connections 2024-04-22T06:32:10.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.429+0000 7f9a9b669640 1 -- 172.21.15.44:0/1573029488 wait complete. 2024-04-22T06:32:10.439 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T06:32:10.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.540+0000 7fba1b070640 1 Processor -- start 2024-04-22T06:32:10.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.540+0000 7fba1b070640 1 -- start start 2024-04-22T06:32:10.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.541+0000 7fba1b070640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fba1415e670 0x7fba1415ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:10.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.541+0000 7fba1b070640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba140cd170 0x7fba140cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:10.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.541+0000 7fba1b070640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba140cb990 con 0x7fba140c7d00 2024-04-22T06:32:10.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.541+0000 7fba1b070640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fba1415f480 con 0x7fba1415e670 2024-04-22T06:32:10.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.541+0000 7fba1b070640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fba141558d0 con 0x7fba140cd170 2024-04-22T06:32:10.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.541+0000 7fba18e0c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fba1415e670 0x7fba1415ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:10.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.541+0000 7fba18e0c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fba1415e670 0x7fba1415ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52644/0 (socket says 172.21.15.44:52644) 2024-04-22T06:32:10.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.541+0000 7fba18e0c640 1 -- 172.21.15.44:0/3241464018 learned_addr learned my addr 172.21.15.44:0/3241464018 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:10.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.541+0000 7fba1960d640 1 --2- 172.21.15.44:0/3241464018 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba140cd170 0x7fba140cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:10.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.541+0000 7fba18e0c640 1 -- 172.21.15.44:0/3241464018 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba140cd170 msgr2=0x7fba140cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:10.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.541+0000 7fba18e0c640 1 --2- 172.21.15.44:0/3241464018 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba140cd170 0x7fba140cd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:10.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.541+0000 7fba18e0c640 1 -- 172.21.15.44:0/3241464018 >> v1:172.21.15.70:6789/0 conn(0x7fba140c7d00 legacy=0x7fba1406d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:10.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.541+0000 7fba18e0c640 1 -- 172.21.15.44:0/3241464018 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba140cdb00 con 0x7fba1415e670 2024-04-22T06:32:10.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.541+0000 7fba1960d640 1 --2- 172.21.15.44:0/3241464018 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba140cd170 0x7fba140cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:32:10.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.542+0000 7fba18e0c640 1 --2- 172.21.15.44:0/3241464018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba1415e670 0x7fba1415ea50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fba0c009e50 tx=0x7fba0c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f886b9a747f631fb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:10.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.542+0000 7fba137fe640 1 -- 172.21.15.44:0/3241464018 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c02c9e0 con 0x7fba1415e670 2024-04-22T06:32:10.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.542+0000 7fba137fe640 1 -- 172.21.15.44:0/3241464018 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba0c02cb80 con 0x7fba1415e670 2024-04-22T06:32:10.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.542+0000 7fba137fe640 1 -- 172.21.15.44:0/3241464018 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c0343f0 con 0x7fba1415e670 2024-04-22T06:32:10.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.543+0000 7fba1b070640 1 -- 172.21.15.44:0/3241464018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba1415e670 msgr2=0x7fba1415ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:10.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.543+0000 7fba1b070640 1 --2- 172.21.15.44:0/3241464018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba1415e670 0x7fba1415ea50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fba0c009e50 tx=0x7fba0c02be40 comp rx=0 tx=0).stop 2024-04-22T06:32:10.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.543+0000 7fba1b070640 1 -- 172.21.15.44:0/3241464018 shutdown_connections 2024-04-22T06:32:10.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.543+0000 7fba1b070640 1 --2- 172.21.15.44:0/3241464018 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba140cd170 0x7fba140cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:10.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.543+0000 7fba1b070640 1 --2- 172.21.15.44:0/3241464018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba1415e670 0x7fba1415ea50 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:10.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.543+0000 7fba1b070640 1 -- 172.21.15.44:0/3241464018 >> 172.21.15.44:0/3241464018 conn(0x7fba14072fb0 msgr2=0x7fba140733c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:10.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.543+0000 7fba1b070640 1 -- 172.21.15.44:0/3241464018 shutdown_connections 2024-04-22T06:32:10.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.543+0000 7fba1b070640 1 -- 172.21.15.44:0/3241464018 wait complete. 2024-04-22T06:32:10.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.544+0000 7fba1b070640 1 Processor -- start 2024-04-22T06:32:10.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.544+0000 7fba1b070640 1 -- start start 2024-04-22T06:32:10.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.545+0000 7fba1b070640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba140c7d00 0x7fba141e4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:10.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.545+0000 7fba1b070640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba140cd170 0x7fba140c1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:10.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.545+0000 7fba1b070640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba1415e670 0x7fba141f18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:10.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.545+0000 7fba1b070640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fba140cc810 con 0x7fba140cd170 2024-04-22T06:32:10.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.545+0000 7fba1b070640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fba140cbf70 con 0x7fba1415e670 2024-04-22T06:32:10.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.545+0000 7fba1b070640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fba140ccaf0 con 0x7fba140c7d00 2024-04-22T06:32:10.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.545+0000 7fba13fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba140cd170 0x7fba140c1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:10.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.545+0000 7fba13fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba140cd170 0x7fba140c1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52656/0 (socket says 172.21.15.44:52656) 2024-04-22T06:32:10.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.545+0000 7fba13fff640 1 -- 172.21.15.44:0/3039247130 learned_addr learned my addr 172.21.15.44:0/3039247130 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:10.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.545+0000 7fba18e0c640 1 --2- 172.21.15.44:0/3039247130 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba140c7d00 0x7fba141e4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:10.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.545+0000 7fba1960d640 1 --2- 172.21.15.44:0/3039247130 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba1415e670 0x7fba141f18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:10.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.546+0000 7fba13fff640 1 -- 172.21.15.44:0/3039247130 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba140c7d00 msgr2=0x7fba141e4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:10.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.546+0000 7fba13fff640 1 --2- 172.21.15.44:0/3039247130 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba140c7d00 0x7fba141e4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:10.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.546+0000 7fba13fff640 1 -- 172.21.15.44:0/3039247130 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba1415e670 msgr2=0x7fba141f18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:10.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.546+0000 7fba13fff640 1 --2- 172.21.15.44:0/3039247130 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba1415e670 0x7fba141f18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:10.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.546+0000 7fba13fff640 1 -- 172.21.15.44:0/3039247130 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba14156250 con 0x7fba140cd170 2024-04-22T06:32:10.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.546+0000 7fba1960d640 1 --2- 172.21.15.44:0/3039247130 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba1415e670 0x7fba141f18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:32:10.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.546+0000 7fba13fff640 1 --2- 172.21.15.44:0/3039247130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba140cd170 0x7fba140c1fc0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc00c430 tx=0x7fb9fc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:10.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.546+0000 7fba117fa640 1 -- 172.21.15.44:0/3039247130 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc011790 con 0x7fba140cd170 2024-04-22T06:32:10.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.546+0000 7fba117fa640 1 -- 172.21.15.44:0/3039247130 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9fc017070 con 0x7fba140cd170 2024-04-22T06:32:10.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.546+0000 7fba117fa640 1 -- 172.21.15.44:0/3039247130 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc011bb0 con 0x7fba140cd170 2024-04-22T06:32:10.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.546+0000 7fba1b070640 1 -- 172.21.15.44:0/3039247130 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba141f5080 con 0x7fba140cd170 2024-04-22T06:32:10.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.546+0000 7fba1b070640 1 -- 172.21.15.44:0/3039247130 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba140c2ca0 con 0x7fba140cd170 2024-04-22T06:32:10.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.547+0000 7fba1b070640 1 -- 172.21.15.44:0/3039247130 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba140b6800 con 0x7fba140cd170 2024-04-22T06:32:10.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.547+0000 7fba117fa640 1 -- 172.21.15.44:0/3039247130 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb9fc01c460 con 0x7fba140cd170 2024-04-22T06:32:10.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.548+0000 7fba117fa640 1 --2- 172.21.15.44:0/3039247130 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb9f406c8f0 0x7fb9f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:10.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.548+0000 7fba117fa640 1 -- 172.21.15.44:0/3039247130 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fb9fc08bce0 con 0x7fba140cd170 2024-04-22T06:32:10.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.548+0000 7fba18e0c640 1 --2- 172.21.15.44:0/3039247130 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb9f406c8f0 0x7fb9f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:10.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.548+0000 7fba18e0c640 1 --2- 172.21.15.44:0/3039247130 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb9f406c8f0 0x7fb9f406edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fba0c035000 tx=0x7fba0c030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:10.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.552+0000 7fba117fa640 1 -- 172.21.15.44:0/3039247130 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fba140b6800 con 0x7fba140cd170 2024-04-22T06:32:10.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.777+0000 7fba1b070640 1 -- 172.21.15.44:0/3039247130 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba140cebc0 con 0x7fba140cd170 2024-04-22T06:32:10.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.778+0000 7fba117fa640 1 -- 172.21.15.44:0/3039247130 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+6905 (secure 0 0 0) 0x7fb9fc059da0 con 0x7fba140cd170 2024-04-22T06:32:10.777 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:32:10.777 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/1779271860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":1779271860},{"type":"v1","addr":"172.21.15.70:6836","nonce":1779271860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T06:31:06.356312+0000","modified":"2024-04-22T06:31:56.168744+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4308,"mds_1":4305,"mds_2":4293,"mds_3":4290,"mds_4":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":4,"incarnation":17,"state":"up:active","state_seq":15,"addr":"172.21.15.44:6834/1497302315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1497302315},{"type":"v1","addr":"172.21.15.44:6834","nonce":1497302315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4290":{"gid":4290,"name":"a","rank":3,"incarnation":15,"state":"up:active","state_seq":14,"addr":"172.21.15.44:6835/2383621014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2383621014},{"type":"v1","addr":"172.21.15.44:6835","nonce":2383621014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"e","rank":2,"incarnation":13,"state":"up:active","state_seq":12,"addr":"172.21.15.44:6837/915960587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":915960587},{"type":"v1","addr":"172.21.15.44:6837","nonce":915960587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4305":{"gid":4305,"name":"b","rank":1,"incarnation":9,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6838/924955785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":924955785},{"type":"v1","addr":"172.21.15.70:6838","nonce":924955785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4308":{"gid":4308,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.70:6839/2915150040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6837","nonce":2915150040},{"type":"v1","addr":"172.21.15.70:6839","nonce":2915150040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4308,"qdb_cluster":[4287,4290,4293,4305,4308]},"id":1}]} 2024-04-22T06:32:10.777 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 18 2024-04-22T06:32:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.780+0000 7fba1b070640 1 -- 172.21.15.44:0/3039247130 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb9f406c8f0 msgr2=0x7fb9f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.780+0000 7fba1b070640 1 --2- 172.21.15.44:0/3039247130 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb9f406c8f0 0x7fb9f406edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fba0c035000 tx=0x7fba0c030000 comp rx=0 tx=0).stop 2024-04-22T06:32:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.780+0000 7fba1b070640 1 -- 172.21.15.44:0/3039247130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba140cd170 msgr2=0x7fba140c1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.780+0000 7fba1b070640 1 --2- 172.21.15.44:0/3039247130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba140cd170 0x7fba140c1fc0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc00c430 tx=0x7fb9fc00c8f0 comp rx=0 tx=0).stop 2024-04-22T06:32:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.780+0000 7fba1b070640 1 -- 172.21.15.44:0/3039247130 shutdown_connections 2024-04-22T06:32:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.780+0000 7fba1b070640 1 --2- 172.21.15.44:0/3039247130 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb9f406c8f0 0x7fb9f406edb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.780+0000 7fba1b070640 1 --2- 172.21.15.44:0/3039247130 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba1415e670 0x7fba141f18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.780+0000 7fba1b070640 1 --2- 172.21.15.44:0/3039247130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba140cd170 0x7fba140c1fc0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.780+0000 7fba1b070640 1 --2- 172.21.15.44:0/3039247130 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba140c7d00 0x7fba141e4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.780+0000 7fba1b070640 1 -- 172.21.15.44:0/3039247130 >> 172.21.15.44:0/3039247130 conn(0x7fba14072fb0 msgr2=0x7fba14073390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.780+0000 7fba1b070640 1 -- 172.21.15.44:0/3039247130 shutdown_connections 2024-04-22T06:32:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.780+0000 7fba1b070640 1 -- 172.21.15.44:0/3039247130 wait complete. 2024-04-22T06:32:10.790 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T06:32:10.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.892+0000 7f6ee9631640 1 Processor -- start 2024-04-22T06:32:10.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.892+0000 7f6ee9631640 1 -- start start 2024-04-22T06:32:10.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.893+0000 7f6ee9631640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ee41552a0 0x7f6ee415bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:10.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.893+0000 7f6ee9631640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ee406c6e0 con 0x7f6ee4155680 2024-04-22T06:32:10.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.893+0000 7f6ee9631640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ee406c9c0 con 0x7f6ee40cc4e0 2024-04-22T06:32:10.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.893+0000 7f6ee9631640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f6ee40cb990 con 0x7f6ee41552a0 2024-04-22T06:32:10.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.893+0000 7f6ee27fc640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f6ee4155680 0x7f6ee40cbdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40210/0 (socket says 172.21.15.44:40210) 2024-04-22T06:32:10.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.893+0000 7f6ee27fc640 1 -- 172.21.15.44:0/1006293881 learned_addr learned my addr 172.21.15.44:0/1006293881 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:32:10.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.893+0000 7f6ee2ffd640 1 --2- 172.21.15.44:0/1006293881 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ee41552a0 0x7f6ee415bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:10.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.893+0000 7f6ee1ffb640 1 -- 172.21.15.44:0/1006293881 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1746699399 0 0) 0x7f6ee406c6e0 con 0x7f6ee4155680 2024-04-22T06:32:10.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.893+0000 7f6ee1ffb640 1 -- 172.21.15.44:0/1006293881 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ecc003520 con 0x7f6ee4155680 2024-04-22T06:32:10.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.893+0000 7f6ee1ffb640 1 -- 172.21.15.44:0/1006293881 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3162976712 0 0) 0x7f6ecc003520 con 0x7f6ee4155680 2024-04-22T06:32:10.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.893+0000 7f6ee1ffb640 1 -- 172.21.15.44:0/1006293881 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ee41552a0 msgr2=0x7f6ee415bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:10.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.893+0000 7f6ee1ffb640 1 --2- 172.21.15.44:0/1006293881 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ee41552a0 0x7f6ee415bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:10.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.893+0000 7f6ee1ffb640 1 -- 172.21.15.44:0/1006293881 >> v1:172.21.15.70:6789/0 conn(0x7f6ee40cc4e0 legacy=0x7f6ee40ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:10.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.893+0000 7f6ee1ffb640 1 -- 172.21.15.44:0/1006293881 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee406cca0 con 0x7f6ee4155680 2024-04-22T06:32:10.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.893+0000 7f6ee1ffb640 1 -- 172.21.15.44:0/1006293881 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1237339257 0 0) 0x7f6ed8001670 con 0x7f6ee4155680 2024-04-22T06:32:10.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.893+0000 7f6ee1ffb640 1 -- 172.21.15.44:0/1006293881 >> v1:172.21.15.44:6789/0 conn(0x7f6ee4155680 legacy=0x7f6ee40cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:10.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.893+0000 7f6ee1ffb640 1 --2- 172.21.15.44:0/1006293881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ecc0039e0 0x7f6ecc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:10.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.893+0000 7f6ee1ffb640 1 --2- 172.21.15.44:0/1006293881 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ee41552a0 0x7f6ecc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:10.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.893+0000 7f6ee1ffb640 1 --2- 172.21.15.44:0/1006293881 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ecc004d40 0x7f6ecc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:10.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.893+0000 7f6ee1ffb640 1 -- 172.21.15.44:0/1006293881 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee40cb990 con 0x7f6ecc0039e0 2024-04-22T06:32:10.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.893+0000 7f6ee1ffb640 1 -- 172.21.15.44:0/1006293881 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc00bb00 con 0x7f6ecc004d40 2024-04-22T06:32:10.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.893+0000 7f6ee1ffb640 1 -- 172.21.15.44:0/1006293881 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f6ecc00bd00 con 0x7f6ee41552a0 2024-04-22T06:32:10.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.894+0000 7f6ee2ffd640 1 --2- 172.21.15.44:0/1006293881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ecc0039e0 0x7f6ecc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:10.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.894+0000 7f6ee27fc640 1 --2- 172.21.15.44:0/1006293881 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ee41552a0 0x7f6ecc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:10.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.894+0000 7f6ee37fe640 1 --2- 172.21.15.44:0/1006293881 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ecc004d40 0x7f6ecc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:10.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.894+0000 7f6ee2ffd640 1 -- 172.21.15.44:0/1006293881 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ee41552a0 msgr2=0x7f6ecc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:10.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.894+0000 7f6ee2ffd640 1 --2- 172.21.15.44:0/1006293881 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ee41552a0 0x7f6ecc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:10.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.894+0000 7f6ee2ffd640 1 -- 172.21.15.44:0/1006293881 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ecc004d40 msgr2=0x7f6ecc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:10.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.894+0000 7f6ee2ffd640 1 --2- 172.21.15.44:0/1006293881 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ecc004d40 0x7f6ecc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:10.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.894+0000 7f6ee2ffd640 1 -- 172.21.15.44:0/1006293881 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ecc003520 con 0x7f6ecc0039e0 2024-04-22T06:32:10.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.894+0000 7f6ee37fe640 1 --2- 172.21.15.44:0/1006293881 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ecc004d40 0x7f6ecc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:32:10.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.895+0000 7f6ee2ffd640 1 --2- 172.21.15.44:0/1006293881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ecc0039e0 0x7f6ecc003e00 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4009a90 tx=0x7f6ecc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2904929534a736 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:10.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.895+0000 7f6ee1ffb640 1 -- 172.21.15.44:0/1006293881 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed4039070 con 0x7f6ecc0039e0 2024-04-22T06:32:10.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.895+0000 7f6ee1ffb640 1 -- 172.21.15.44:0/1006293881 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ed402c680 con 0x7f6ecc0039e0 2024-04-22T06:32:10.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.895+0000 7f6ee1ffb640 1 -- 172.21.15.44:0/1006293881 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed4009690 con 0x7f6ecc0039e0 2024-04-22T06:32:10.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.895+0000 7f6ee9631640 1 -- 172.21.15.44:0/1006293881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ecc0039e0 msgr2=0x7f6ecc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:10.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.895+0000 7f6ee9631640 1 --2- 172.21.15.44:0/1006293881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ecc0039e0 0x7f6ecc003e00 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4009a90 tx=0x7f6ecc005830 comp rx=0 tx=0).stop 2024-04-22T06:32:10.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.895+0000 7f6ee37fe640 1 -- 172.21.15.44:0/1006293881 reap_dead start 2024-04-22T06:32:10.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.895+0000 7f6ee9631640 1 -- 172.21.15.44:0/1006293881 shutdown_connections 2024-04-22T06:32:10.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.895+0000 7f6ee9631640 1 -- 172.21.15.44:0/1006293881 >> 172.21.15.44:0/1006293881 conn(0x7f6ee4075820 msgr2=0x7f6ee4073000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:10.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.896+0000 7f6ee9631640 1 -- 172.21.15.44:0/1006293881 shutdown_connections 2024-04-22T06:32:10.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.896+0000 7f6ee9631640 1 -- 172.21.15.44:0/1006293881 wait complete. 2024-04-22T06:32:10.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.897+0000 7f6ee9631640 1 Processor -- start 2024-04-22T06:32:10.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.897+0000 7f6ee9631640 1 -- start start 2024-04-22T06:32:10.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.897+0000 7f6ee9631640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ecc0039e0 0x7f6ee4159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:10.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.898+0000 7f6ee9631640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ee415bc80 0x7f6ee4156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:10.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.898+0000 7f6ee9631640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ee4158840 0x7f6ee4157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:10.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.898+0000 7f6ee9631640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee406d540 con 0x7f6ee415bc80 2024-04-22T06:32:10.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.898+0000 7f6ee9631640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee406d260 con 0x7f6ee4158840 2024-04-22T06:32:10.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.898+0000 7f6ee9631640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f6ee40b6690 con 0x7f6ecc0039e0 2024-04-22T06:32:10.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.898+0000 7f6ee27fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ee415bc80 0x7f6ee4156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:10.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.898+0000 7f6ee27fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ee415bc80 0x7f6ee4156fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52672/0 (socket says 172.21.15.44:52672) 2024-04-22T06:32:10.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.898+0000 7f6ee27fc640 1 -- 172.21.15.44:0/1952233285 learned_addr learned my addr 172.21.15.44:0/1952233285 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:10.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.898+0000 7f6ee2ffd640 1 --2- 172.21.15.44:0/1952233285 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ecc0039e0 0x7f6ee4159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:10.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.898+0000 7f6ee37fe640 1 --2- 172.21.15.44:0/1952233285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ee4158840 0x7f6ee4157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:10.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.898+0000 7f6ee27fc640 1 -- 172.21.15.44:0/1952233285 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ecc0039e0 msgr2=0x7f6ee4159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:10.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.898+0000 7f6ee27fc640 1 --2- 172.21.15.44:0/1952233285 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ecc0039e0 0x7f6ee4159f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:10.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.898+0000 7f6ee27fc640 1 -- 172.21.15.44:0/1952233285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ee4158840 msgr2=0x7f6ee4157500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:10.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.898+0000 7f6ee27fc640 1 --2- 172.21.15.44:0/1952233285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ee4158840 0x7f6ee4157500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:10.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.898+0000 7f6ee27fc640 1 -- 172.21.15.44:0/1952233285 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee41f0ac0 con 0x7f6ee415bc80 2024-04-22T06:32:10.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.898+0000 7f6ee27fc640 1 --2- 172.21.15.44:0/1952233285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ee415bc80 0x7f6ee4156fc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80072c0 tx=0x7f6ed8007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:10.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.898+0000 7f6ecb7fe640 1 -- 172.21.15.44:0/1952233285 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed800f9c0 con 0x7f6ee415bc80 2024-04-22T06:32:10.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.898+0000 7f6ecb7fe640 1 -- 172.21.15.44:0/1952233285 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ed8018070 con 0x7f6ee415bc80 2024-04-22T06:32:10.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.898+0000 7f6ee9631640 1 -- 172.21.15.44:0/1952233285 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ee41f3da0 con 0x7f6ee415bc80 2024-04-22T06:32:10.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.898+0000 7f6ecb7fe640 1 -- 172.21.15.44:0/1952233285 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed800fdc0 con 0x7f6ee415bc80 2024-04-22T06:32:10.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.898+0000 7f6ee9631640 1 -- 172.21.15.44:0/1952233285 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ee41f4360 con 0x7f6ee415bc80 2024-04-22T06:32:10.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.899+0000 7f6ee9631640 1 -- 172.21.15.44:0/1952233285 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ee41f4a80 con 0x7f6ee415bc80 2024-04-22T06:32:10.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.900+0000 7f6ecb7fe640 1 -- 172.21.15.44:0/1952233285 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6ed801d460 con 0x7f6ee415bc80 2024-04-22T06:32:10.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.900+0000 7f6ecb7fe640 1 --2- 172.21.15.44:0/1952233285 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f6ed006c920 0x7f6ed006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:10.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.900+0000 7f6ecb7fe640 1 -- 172.21.15.44:0/1952233285 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f6ed808c0c0 con 0x7f6ee415bc80 2024-04-22T06:32:10.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.900+0000 7f6ee2ffd640 1 --2- 172.21.15.44:0/1952233285 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f6ed006c920 0x7f6ed006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:10.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.901+0000 7f6ee2ffd640 1 --2- 172.21.15.44:0/1952233285 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f6ed006c920 0x7f6ed006ede0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4036000 tx=0x7f6ed4030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:10.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:10.904+0000 7f6ecb7fe640 1 -- 172.21.15.44:0/1952233285 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6ed805a0f0 con 0x7f6ee415bc80 2024-04-22T06:32:11.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.092+0000 7f6ee9631640 1 -- 172.21.15.44:0/1952233285 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6ee4068cc0 con 0x7f6ee415bc80 2024-04-22T06:32:11.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.093+0000 7f6ecb7fe640 1 -- 172.21.15.44:0/1952233285 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18794 (secure 0 0 0) 0x7f6ed80599f0 con 0x7f6ee415bc80 2024-04-22T06:32:11.092 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:32:11.092 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":27,"fsid":"9b0a34ca-403d-4513-8a66-d3749ba559aa","created":"2024-04-22T06:30:42.334581+0000","modified":"2024-04-22T06:31:48.138913+0000","last_up_change":"2024-04-22T06:30:52.942257+0000","last_in_change":"2024-04-22T06:30:44.467727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T06:30:54.560256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T06:31:04.612387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T06:31:05.312124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-22T06:31:09.025550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b4a6692-53c9-472d-b2f3-1dd29a6679ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6817","nonce":2220268493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6819","nonce":2220268493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6823","nonce":2220268493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6821","nonce":2220268493}]},"public_addr":"172.21.15.44:6817/2220268493","cluster_addr":"172.21.15.44:6819/2220268493","heartbeat_back_addr":"172.21.15.44:6823/2220268493","heartbeat_front_addr":"172.21.15.44:6821/2220268493","state":["exists","up"]},{"osd":1,"uuid":"70a6370a-7c0a-4d64-a0e4-b2e726b4f291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6809","nonce":1278678977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6811","nonce":1278678977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6815","nonce":1278678977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6813","nonce":1278678977}]},"public_addr":"172.21.15.44:6809/1278678977","cluster_addr":"172.21.15.44:6811/1278678977","heartbeat_back_addr":"172.21.15.44:6815/1278678977","heartbeat_front_addr":"172.21.15.44:6813/1278678977","state":["exists","up"]},{"osd":2,"uuid":"f2e51a62-ca91-470d-bd5a-5710c61deee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6801","nonce":2544087903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6803","nonce":2544087903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6807","nonce":2544087903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6805","nonce":2544087903}]},"public_addr":"172.21.15.44:6801/2544087903","cluster_addr":"172.21.15.44:6803/2544087903","heartbeat_back_addr":"172.21.15.44:6807/2544087903","heartbeat_front_addr":"172.21.15.44:6805/2544087903","state":["exists","up"]},{"osd":3,"uuid":"bc514203-9af9-4f1f-b18e-765bdd5de6e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6825","nonce":1692928109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6827","nonce":1692928109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6831","nonce":1692928109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6829","nonce":1692928109}]},"public_addr":"172.21.15.44:6825/1692928109","cluster_addr":"172.21.15.44:6827/1692928109","heartbeat_back_addr":"172.21.15.44:6831/1692928109","heartbeat_front_addr":"172.21.15.44:6829/1692928109","state":["exists","up"]},{"osd":4,"uuid":"357c439a-6e08-4c19-ac18-b68e6c890497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6820","nonce":3802577427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6824","nonce":3802577427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6831","nonce":3802577427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6828","nonce":3802577427}]},"public_addr":"172.21.15.70:6820/3802577427","cluster_addr":"172.21.15.70:6824/3802577427","heartbeat_back_addr":"172.21.15.70:6831/3802577427","heartbeat_front_addr":"172.21.15.70:6828/3802577427","state":["exists","up"]},{"osd":5,"uuid":"952fb88f-d56e-4305-bfa8-d32674332dfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6801","nonce":664757345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6803","nonce":664757345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6807","nonce":664757345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6805","nonce":664757345}]},"public_addr":"172.21.15.70:6801/664757345","cluster_addr":"172.21.15.70:6803/664757345","heartbeat_back_addr":"172.21.15.70:6807/664757345","heartbeat_front_addr":"172.21.15.70:6805/664757345","state":["exists","up"]},{"osd":6,"uuid":"74399b3d-fac2-45ab-9172-2cd95b02450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6817","nonce":1623253482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6821","nonce":1623253482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6827","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6829","nonce":1623253482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6825","nonce":1623253482}]},"public_addr":"172.21.15.70:6817/1623253482","cluster_addr":"172.21.15.70:6821/1623253482","heartbeat_back_addr":"172.21.15.70:6829/1623253482","heartbeat_front_addr":"172.21.15.70:6825/1623253482","state":["exists","up"]},{"osd":7,"uuid":"8f35508d-6fcc-43ff-8ee7-0498a7ab3b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6809","nonce":1551180300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6811","nonce":1551180300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6815","nonce":1551180300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6813","nonce":1551180300}]},"public_addr":"172.21.15.70:6809/1551180300","cluster_addr":"172.21.15.70:6811/1551180300","heartbeat_back_addr":"172.21.15.70:6815/1551180300","heartbeat_front_addr":"172.21.15.70:6813/1551180300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.612502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.653756+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.521169+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.649955+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.640231+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.686638+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.642199+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.700360+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T06:32:11.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.095+0000 7f6ee9631640 1 -- 172.21.15.44:0/1952233285 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f6ed006c920 msgr2=0x7f6ed006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:11.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.095+0000 7f6ee9631640 1 --2- 172.21.15.44:0/1952233285 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f6ed006c920 0x7f6ed006ede0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4036000 tx=0x7f6ed4030000 comp rx=0 tx=0).stop 2024-04-22T06:32:11.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.095+0000 7f6ee9631640 1 -- 172.21.15.44:0/1952233285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ee415bc80 msgr2=0x7f6ee4156fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:11.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.095+0000 7f6ee9631640 1 --2- 172.21.15.44:0/1952233285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ee415bc80 0x7f6ee4156fc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80072c0 tx=0x7f6ed8007780 comp rx=0 tx=0).stop 2024-04-22T06:32:11.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.095+0000 7f6ee9631640 1 -- 172.21.15.44:0/1952233285 shutdown_connections 2024-04-22T06:32:11.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.095+0000 7f6ee9631640 1 --2- 172.21.15.44:0/1952233285 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f6ed006c920 0x7f6ed006ede0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:11.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.095+0000 7f6ee9631640 1 --2- 172.21.15.44:0/1952233285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ee4158840 0x7f6ee4157500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:11.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.095+0000 7f6ee9631640 1 --2- 172.21.15.44:0/1952233285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ee415bc80 0x7f6ee4156fc0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:11.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.095+0000 7f6ee9631640 1 --2- 172.21.15.44:0/1952233285 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ecc0039e0 0x7f6ee4159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:11.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.095+0000 7f6ee9631640 1 -- 172.21.15.44:0/1952233285 >> 172.21.15.44:0/1952233285 conn(0x7f6ee4075820 msgr2=0x7f6ee4072a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:11.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.095+0000 7f6ee9631640 1 -- 172.21.15.44:0/1952233285 shutdown_connections 2024-04-22T06:32:11.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.095+0000 7f6ee9631640 1 -- 172.21.15.44:0/1952233285 wait complete. 2024-04-22T06:32:11.105 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with random seed 418533 2024-04-22T06:32:11.105 DEBUG:tasks.quiescer:Yielding 2024-04-22T06:32:11.105 INFO:teuthology.run_tasks:Running task workunit... 2024-04-22T06:32:11.119 INFO:tasks.workunit:Pulling workunits from ref 86f7587a5a09af35f0895e1a2d08527638fae697 2024-04-22T06:32:11.120 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-22T06:32:11.120 INFO:tasks.workunit:timeout=3h 2024-04-22T06:32:11.120 INFO:tasks.workunit:cleanup=True 2024-04-22T06:32:11.121 DEBUG:teuthology.orchestra.run.smithi044:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-22T06:32:11.121 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T06:32:11.137 INFO:teuthology.orchestra.run.smithi044.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-22T06:32:11.137 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-22T06:32:11.137 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 2bh/43d Inode: 1 Links: 2 2024-04-22T06:32:11.137 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T06:32:11.137 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-22T06:32:11.137 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-22T06:32:11.137 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-04-22 06:31:12.009482698 +0000 2024-04-22T06:32:11.137 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-04-22 06:31:19.270819439 +0000 2024-04-22T06:32:11.137 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: - 2024-04-22T06:32:11.138 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-22T06:32:11.138 DEBUG:teuthology.orchestra.run.smithi044:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-22T06:32:11.208 DEBUG:teuthology.orchestra.run.smithi044:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 86f7587a5a09af35f0895e1a2d08527638fae697 2024-04-22T06:32:11.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.225+0000 7fb9f004e640 1 Processor -- start 2024-04-22T06:32:11.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.226+0000 7fb9f004e640 1 -- start start 2024-04-22T06:32:11.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.226+0000 7fb9f004e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9e815f0d0 0x7fb9e80d0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:11.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.226+0000 7fb9f004e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb9e80d1400 0x7fb9e80cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:11.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.227+0000 7fb9f004e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9e815b200 con 0x7fb9e8155240 2024-04-22T06:32:11.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.227+0000 7fb9f004e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fb9e815c060 con 0x7fb9e80d1400 2024-04-22T06:32:11.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.227+0000 7fb9f004e640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb9e815b4e0 con 0x7fb9e815f0d0 2024-04-22T06:32:11.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.227+0000 7fb9ee5eb640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb9e80d1400 0x7fb9e80cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:11.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.227+0000 7fb9ed5e9640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9e815f0d0 0x7fb9e80d0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:11.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.227+0000 7fb9eddea640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb9e8155240 0x7fb9e80cb320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.44:51200/0 (socket says 172.21.15.44:51200) 2024-04-22T06:32:11.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.227+0000 7fb9ee5eb640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb9e80d1400 0x7fb9e80cc480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:46426/0 (socket says 172.21.15.44:46426) 2024-04-22T06:32:11.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.227+0000 7fb9eddea640 1 -- 172.21.15.44:0/326817783 learned_addr learned my addr 172.21.15.44:0/326817783 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:32:11.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.227+0000 7fb9ee5eb640 1 -- 172.21.15.44:0/326817783 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9e815f0d0 msgr2=0x7fb9e80d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:11.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.227+0000 7fb9ee5eb640 1 --2- 172.21.15.44:0/326817783 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9e815f0d0 0x7fb9e80d0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:11.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.227+0000 7fb9ee5eb640 1 -- 172.21.15.44:0/326817783 >> v1:172.21.15.70:6789/0 conn(0x7fb9e8155240 legacy=0x7fb9e80cb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:11.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.227+0000 7fb9ee5eb640 1 -- 172.21.15.44:0/326817783 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9e815b7c0 con 0x7fb9e80d1400 2024-04-22T06:32:11.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.227+0000 7fb9ed5e9640 1 --2- 172.21.15.44:0/326817783 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9e815f0d0 0x7fb9e80d0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:32:11.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.228+0000 7fb9ee5eb640 1 --2- 172.21.15.44:0/326817783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9e80d1400 0x7fb9e80cc480 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0009d90 tx=0x7fb9e002bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=735991c9ad7fff4f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:11.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.228+0000 7fb9ecde8640 1 -- 172.21.15.44:0/326817783 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e0039070 con 0x7fb9e80d1400 2024-04-22T06:32:11.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.228+0000 7fb9ecde8640 1 -- 172.21.15.44:0/326817783 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9e002cc50 con 0x7fb9e80d1400 2024-04-22T06:32:11.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.228+0000 7fb9ecde8640 1 -- 172.21.15.44:0/326817783 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e0034440 con 0x7fb9e80d1400 2024-04-22T06:32:11.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.229+0000 7fb9f004e640 1 -- 172.21.15.44:0/326817783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9e80d1400 msgr2=0x7fb9e80cc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:11.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.229+0000 7fb9f004e640 1 --2- 172.21.15.44:0/326817783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9e80d1400 0x7fb9e80cc480 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0009d90 tx=0x7fb9e002bea0 comp rx=0 tx=0).stop 2024-04-22T06:32:11.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.229+0000 7fb9f004e640 1 -- 172.21.15.44:0/326817783 shutdown_connections 2024-04-22T06:32:11.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.229+0000 7fb9f004e640 1 --2- 172.21.15.44:0/326817783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9e80d1400 0x7fb9e80cc480 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:11.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.229+0000 7fb9f004e640 1 --2- 172.21.15.44:0/326817783 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9e815f0d0 0x7fb9e80d0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:11.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.229+0000 7fb9f004e640 1 -- 172.21.15.44:0/326817783 >> 172.21.15.44:0/326817783 conn(0x7fb9e80750d0 msgr2=0x7fb9e80754e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:11.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.229+0000 7fb9f004e640 1 -- 172.21.15.44:0/326817783 shutdown_connections 2024-04-22T06:32:11.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.229+0000 7fb9f004e640 1 -- 172.21.15.44:0/326817783 wait complete. 2024-04-22T06:32:11.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.230+0000 7fb9f004e640 1 Processor -- start 2024-04-22T06:32:11.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.230+0000 7fb9f004e640 1 -- start start 2024-04-22T06:32:11.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.231+0000 7fb9f004e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9e80d1400 0x7fb9e81f4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:11.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.231+0000 7fb9f004e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9e8155240 0x7fb9e814ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:11.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.231+0000 7fb9f004e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9e815f0d0 0x7fb9e8145180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:11.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.231+0000 7fb9f004e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e815f4f0 con 0x7fb9e8155240 2024-04-22T06:32:11.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.231+0000 7fb9f004e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e815bd80 con 0x7fb9e80d1400 2024-04-22T06:32:11.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.231+0000 7fb9f004e640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb9e8155620 con 0x7fb9e815f0d0 2024-04-22T06:32:11.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.231+0000 7fb9ed5e9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9e8155240 0x7fb9e814ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:11.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.231+0000 7fb9ed5e9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9e8155240 0x7fb9e814ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52682/0 (socket says 172.21.15.44:52682) 2024-04-22T06:32:11.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.231+0000 7fb9ed5e9640 1 -- 172.21.15.44:0/906954413 learned_addr learned my addr 172.21.15.44:0/906954413 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:11.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.231+0000 7fb9eddea640 1 --2- 172.21.15.44:0/906954413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9e80d1400 0x7fb9e81f4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:11.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.231+0000 7fb9ee5eb640 1 --2- 172.21.15.44:0/906954413 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9e815f0d0 0x7fb9e8145180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:11.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.232+0000 7fb9ed5e9640 1 -- 172.21.15.44:0/906954413 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9e815f0d0 msgr2=0x7fb9e8145180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:11.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.232+0000 7fb9ed5e9640 1 --2- 172.21.15.44:0/906954413 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9e815f0d0 0x7fb9e8145180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:11.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.232+0000 7fb9ed5e9640 1 -- 172.21.15.44:0/906954413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9e80d1400 msgr2=0x7fb9e81f4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:11.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.232+0000 7fb9ed5e9640 1 --2- 172.21.15.44:0/906954413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9e80d1400 0x7fb9e81f4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:11.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.232+0000 7fb9ed5e9640 1 -- 172.21.15.44:0/906954413 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9e81f51c0 con 0x7fb9e8155240 2024-04-22T06:32:11.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.232+0000 7fb9ed5e9640 1 --2- 172.21.15.44:0/906954413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9e8155240 0x7fb9e814ada0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb9d800ebe0 tx=0x7fb9d800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:11.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.232+0000 7fb9de7fc640 1 -- 172.21.15.44:0/906954413 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8017070 con 0x7fb9e8155240 2024-04-22T06:32:11.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.232+0000 7fb9de7fc640 1 -- 172.21.15.44:0/906954413 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9d800cc30 con 0x7fb9e8155240 2024-04-22T06:32:11.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.232+0000 7fb9f004e640 1 -- 172.21.15.44:0/906954413 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9e80b48c0 con 0x7fb9e8155240 2024-04-22T06:32:11.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.232+0000 7fb9de7fc640 1 -- 172.21.15.44:0/906954413 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8013470 con 0x7fb9e8155240 2024-04-22T06:32:11.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.232+0000 7fb9f004e640 1 -- 172.21.15.44:0/906954413 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9e80cedf0 con 0x7fb9e8155240 2024-04-22T06:32:11.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.233+0000 7fb9f004e640 1 -- 172.21.15.44:0/906954413 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9e8068cc0 con 0x7fb9e8155240 2024-04-22T06:32:11.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.234+0000 7fb9de7fc640 1 -- 172.21.15.44:0/906954413 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb9d801a070 con 0x7fb9e8155240 2024-04-22T06:32:11.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.234+0000 7fb9de7fc640 1 --2- 172.21.15.44:0/906954413 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb9cc06c8f0 0x7fb9cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:11.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.234+0000 7fb9de7fc640 1 -- 172.21.15.44:0/906954413 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fb9d808bfc0 con 0x7fb9e8155240 2024-04-22T06:32:11.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.235+0000 7fb9eddea640 1 --2- 172.21.15.44:0/906954413 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb9cc06c8f0 0x7fb9cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:11.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.235+0000 7fb9eddea640 1 --2- 172.21.15.44:0/906954413 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb9cc06c8f0 0x7fb9cc06edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4007fd0 tx=0x7fb9e4007e80 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:11.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.239+0000 7fb9de7fc640 1 -- 172.21.15.44:0/906954413 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb9d80597f0 con 0x7fb9e8155240 2024-04-22T06:32:11.240 INFO:tasks.workunit.client.0.smithi044.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-22T06:32:11.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.468+0000 7fb9f004e640 1 -- 172.21.15.44:0/906954413 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb9e814bbf0 con 0x7fb9e8155240 2024-04-22T06:32:11.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.468+0000 7fb9de7fc640 1 -- 172.21.15.44:0/906954413 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+6905 (secure 0 0 0) 0x7fb9d805daf0 con 0x7fb9e8155240 2024-04-22T06:32:11.467 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:32:11.467 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/1779271860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":1779271860},{"type":"v1","addr":"172.21.15.70:6836","nonce":1779271860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T06:31:06.356312+0000","modified":"2024-04-22T06:31:56.168744+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4308,"mds_1":4305,"mds_2":4293,"mds_3":4290,"mds_4":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":4,"incarnation":17,"state":"up:active","state_seq":15,"addr":"172.21.15.44:6834/1497302315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1497302315},{"type":"v1","addr":"172.21.15.44:6834","nonce":1497302315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4290":{"gid":4290,"name":"a","rank":3,"incarnation":15,"state":"up:active","state_seq":14,"addr":"172.21.15.44:6835/2383621014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2383621014},{"type":"v1","addr":"172.21.15.44:6835","nonce":2383621014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"e","rank":2,"incarnation":13,"state":"up:active","state_seq":12,"addr":"172.21.15.44:6837/915960587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":915960587},{"type":"v1","addr":"172.21.15.44:6837","nonce":915960587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4305":{"gid":4305,"name":"b","rank":1,"incarnation":9,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6838/924955785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":924955785},{"type":"v1","addr":"172.21.15.70:6838","nonce":924955785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4308":{"gid":4308,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.70:6839/2915150040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6837","nonce":2915150040},{"type":"v1","addr":"172.21.15.70:6839","nonce":2915150040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4308,"qdb_cluster":[4287,4290,4293,4305,4308]},"id":1}]} 2024-04-22T06:32:11.467 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 18 2024-04-22T06:32:11.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.470+0000 7fb9f004e640 1 -- 172.21.15.44:0/906954413 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb9cc06c8f0 msgr2=0x7fb9cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:11.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.470+0000 7fb9f004e640 1 --2- 172.21.15.44:0/906954413 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb9cc06c8f0 0x7fb9cc06edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4007fd0 tx=0x7fb9e4007e80 comp rx=0 tx=0).stop 2024-04-22T06:32:11.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.470+0000 7fb9f004e640 1 -- 172.21.15.44:0/906954413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9e8155240 msgr2=0x7fb9e814ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:11.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.470+0000 7fb9f004e640 1 --2- 172.21.15.44:0/906954413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9e8155240 0x7fb9e814ada0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb9d800ebe0 tx=0x7fb9d800c520 comp rx=0 tx=0).stop 2024-04-22T06:32:11.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.470+0000 7fb9f004e640 1 -- 172.21.15.44:0/906954413 shutdown_connections 2024-04-22T06:32:11.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.470+0000 7fb9f004e640 1 --2- 172.21.15.44:0/906954413 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb9cc06c8f0 0x7fb9cc06edb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:11.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.470+0000 7fb9f004e640 1 --2- 172.21.15.44:0/906954413 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9e815f0d0 0x7fb9e8145180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:11.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.470+0000 7fb9f004e640 1 --2- 172.21.15.44:0/906954413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9e8155240 0x7fb9e814ada0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:11.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.470+0000 7fb9f004e640 1 --2- 172.21.15.44:0/906954413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9e80d1400 0x7fb9e81f4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:11.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.470+0000 7fb9f004e640 1 -- 172.21.15.44:0/906954413 >> 172.21.15.44:0/906954413 conn(0x7fb9e80750d0 msgr2=0x7fb9e80cc380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:11.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.470+0000 7fb9f004e640 1 -- 172.21.15.44:0/906954413 shutdown_connections 2024-04-22T06:32:11.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:11.470+0000 7fb9f004e640 1 -- 172.21.15.44:0/906954413 wait complete. 2024-04-22T06:32:12.084 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-22T06:31:06.356312+0000', 'modified': '2024-04-22T06:31:56.168744+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 4308, 'mds_1': 4305, 'mds_2': 4293, 'mds_3': 4290, 'mds_4': 4287}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4287': {'gid': 4287, 'name': 'c', 'rank': 4, 'incarnation': 17, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.44:6834/1497302315', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6832', 'nonce': 1497302315}, {'type': 'v1', 'addr': '172.21.15.44:6834', 'nonce': 1497302315}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4290': {'gid': 4290, 'name': 'a', 'rank': 3, 'incarnation': 15, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.44:6835/2383621014', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6833', 'nonce': 2383621014}, {'type': 'v1', 'addr': '172.21.15.44:6835', 'nonce': 2383621014}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4293': {'gid': 4293, 'name': 'e', 'rank': 2, 'incarnation': 13, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.44:6837/915960587', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6836', 'nonce': 915960587}, {'type': 'v1', 'addr': '172.21.15.44:6837', 'nonce': 915960587}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4305': {'gid': 4305, 'name': 'b', 'rank': 1, 'incarnation': 9, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.70:6838/924955785', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6834', 'nonce': 924955785}, {'type': 'v1', 'addr': '172.21.15.70:6838', 'nonce': 924955785}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4308': {'gid': 4308, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.70:6839/2915150040', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6837', 'nonce': 2915150040}, {'type': 'v1', 'addr': '172.21.15.70:6839', 'nonce': 2915150040}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4308, 'qdb_cluster': [4287, 4290, 4293, 4305, 4308]} 2024-04-22T06:32:12.085 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-22T06:32:12.085 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-22T06:32:12.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.194+0000 7f2b288e0640 1 Processor -- start 2024-04-22T06:32:12.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.194+0000 7f2b288e0640 1 -- start start 2024-04-22T06:32:12.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.195+0000 7f2b288e0640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2b240c5c00 0x7f2b240c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.195+0000 7f2b288e0640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b240ccde0 con 0x7f2b2415f050 2024-04-22T06:32:12.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.195+0000 7f2b288e0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b240cd0a0 con 0x7f2b24155460 2024-04-22T06:32:12.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.195+0000 7f2b288e0640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f2b24155840 con 0x7f2b240c5c00 2024-04-22T06:32:12.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.195+0000 7f2b22d9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2b240c5c00 0x7f2b240c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.195+0000 7f2b21d9b640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f2b2415f050 0x7f2b24073230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:55530/0 (socket says 172.21.15.44:55530) 2024-04-22T06:32:12.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.195+0000 7f2b22d9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2b240c5c00 0x7f2b240c6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36536/0 (socket says 172.21.15.44:36536) 2024-04-22T06:32:12.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.195+0000 7f2b21d9b640 1 -- 172.21.15.44:0/82792430 learned_addr learned my addr 172.21.15.44:0/82792430 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:32:12.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.195+0000 7f2b22d9d640 1 -- 172.21.15.44:0/82792430 >> v1:172.21.15.70:6789/0 conn(0x7f2b24155460 legacy=0x7f2b240cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.195+0000 7f2b22d9d640 1 -- 172.21.15.44:0/82792430 >> v1:172.21.15.44:6789/0 conn(0x7f2b2415f050 legacy=0x7f2b24073230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.195+0000 7f2b22d9d640 1 -- 172.21.15.44:0/82792430 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b240d0910 con 0x7f2b240c5c00 2024-04-22T06:32:12.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.196+0000 7f2b22d9d640 1 --2- 172.21.15.44:0/82792430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b240c5c00 0x7f2b240c6010 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c009d90 tx=0x7f2b1c02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5eddd86ef21798a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:12.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.196+0000 7f2b2159a640 1 -- 172.21.15.44:0/82792430 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c039070 con 0x7f2b240c5c00 2024-04-22T06:32:12.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.196+0000 7f2b2159a640 1 -- 172.21.15.44:0/82792430 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b1c02cd00 con 0x7f2b240c5c00 2024-04-22T06:32:12.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.196+0000 7f2b2159a640 1 -- 172.21.15.44:0/82792430 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c034660 con 0x7f2b240c5c00 2024-04-22T06:32:12.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.196+0000 7f2b288e0640 1 -- 172.21.15.44:0/82792430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b240c5c00 msgr2=0x7f2b240c6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.196+0000 7f2b288e0640 1 --2- 172.21.15.44:0/82792430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b240c5c00 0x7f2b240c6010 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c009d90 tx=0x7f2b1c02bf50 comp rx=0 tx=0).stop 2024-04-22T06:32:12.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.197+0000 7f2b288e0640 1 -- 172.21.15.44:0/82792430 shutdown_connections 2024-04-22T06:32:12.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.197+0000 7f2b288e0640 1 --2- 172.21.15.44:0/82792430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b240c5c00 0x7f2b240c6010 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.197+0000 7f2b288e0640 1 -- 172.21.15.44:0/82792430 >> 172.21.15.44:0/82792430 conn(0x7f2b2406cca0 msgr2=0x7f2b2406d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:12.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.197+0000 7f2b288e0640 1 -- 172.21.15.44:0/82792430 shutdown_connections 2024-04-22T06:32:12.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.197+0000 7f2b288e0640 1 -- 172.21.15.44:0/82792430 wait complete. 2024-04-22T06:32:12.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.198+0000 7f2b288e0640 1 Processor -- start 2024-04-22T06:32:12.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.198+0000 7f2b288e0640 1 -- start start 2024-04-22T06:32:12.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.199+0000 7f2b288e0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b240c5c00 0x7f2b240cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.199+0000 7f2b288e0640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b24155460 0x7f2b241f1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.199+0000 7f2b288e0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b2415f050 0x7f2b240cf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.199+0000 7f2b288e0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2b240d0ed0 con 0x7f2b240c5c00 2024-04-22T06:32:12.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.199+0000 7f2b288e0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2b240d1520 con 0x7f2b2415f050 2024-04-22T06:32:12.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.199+0000 7f2b288e0640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2b240d11b0 con 0x7f2b24155460 2024-04-22T06:32:12.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.199+0000 7f2b2259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b240c5c00 0x7f2b240cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.199+0000 7f2b2259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b240c5c00 0x7f2b240cfea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36542/0 (socket says 172.21.15.44:36542) 2024-04-22T06:32:12.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.199+0000 7f2b2259c640 1 -- 172.21.15.44:0/4128731766 learned_addr learned my addr 172.21.15.44:0/4128731766 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:12.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.199+0000 7f2b2259c640 1 -- 172.21.15.44:0/4128731766 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b24155460 msgr2=0x7f2b241f1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.199+0000 7f2b21d9b640 1 --2- 172.21.15.44:0/4128731766 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b24155460 0x7f2b241f1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.199+0000 7f2b22d9d640 1 --2- 172.21.15.44:0/4128731766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b2415f050 0x7f2b240cf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.199+0000 7f2b2259c640 1 --2- 172.21.15.44:0/4128731766 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b24155460 0x7f2b241f1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.199+0000 7f2b2259c640 1 -- 172.21.15.44:0/4128731766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b2415f050 msgr2=0x7f2b240cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.199+0000 7f2b2259c640 1 --2- 172.21.15.44:0/4128731766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b2415f050 0x7f2b240cf3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.199+0000 7f2b2259c640 1 -- 172.21.15.44:0/4128731766 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b240d04e0 con 0x7f2b240c5c00 2024-04-22T06:32:12.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.199+0000 7f2b2259c640 1 --2- 172.21.15.44:0/4128731766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b240c5c00 0x7f2b240cfea0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2b1400c400 tx=0x7f2b1400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:12.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.199+0000 7f2b12ffd640 1 -- 172.21.15.44:0/4128731766 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b140052d0 con 0x7f2b240c5c00 2024-04-22T06:32:12.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.199+0000 7f2b12ffd640 1 -- 172.21.15.44:0/4128731766 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b14005490 con 0x7f2b240c5c00 2024-04-22T06:32:12.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.199+0000 7f2b288e0640 1 -- 172.21.15.44:0/4128731766 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b241486a0 con 0x7f2b240c5c00 2024-04-22T06:32:12.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.199+0000 7f2b12ffd640 1 -- 172.21.15.44:0/4128731766 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b14007260 con 0x7f2b240c5c00 2024-04-22T06:32:12.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.199+0000 7f2b288e0640 1 -- 172.21.15.44:0/4128731766 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b24148c60 con 0x7f2b240c5c00 2024-04-22T06:32:12.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.201+0000 7f2b12ffd640 1 -- 172.21.15.44:0/4128731766 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2b14007400 con 0x7f2b240c5c00 2024-04-22T06:32:12.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.201+0000 7f2b12ffd640 1 --2- 172.21.15.44:0/4128731766 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f2b0406c8f0 0x7f2b0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.201+0000 7f2b12ffd640 1 -- 172.21.15.44:0/4128731766 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f2b1408b490 con 0x7f2b240c5c00 2024-04-22T06:32:12.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.201+0000 7f2b21d9b640 1 --2- 172.21.15.44:0/4128731766 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f2b0406c8f0 0x7f2b0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.202+0000 7f2b21d9b640 1 --2- 172.21.15.44:0/4128731766 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f2b0406c8f0 0x7f2b0406edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2b2414a5a0 tx=0x7f2b18009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:12.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.202+0000 7f2b2b5af740 1 Processor -- start 2024-04-22T06:32:12.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.203+0000 7f2b2b5af740 1 -- start start 2024-04-22T06:32:12.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.203+0000 7f2b2b5af740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5624ccf93e40 0x5624ccf9cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.203+0000 7f2b2b5af740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624ccf54ec0 0x5624ccf9d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.203+0000 7f2b2b5af740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5624ccf50ca0 0x5624ccf78ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.203+0000 7f2b2b5af740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5624cceb7b10 con 0x5624ccf54ec0 2024-04-22T06:32:12.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.203+0000 7f2b2b5af740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5624cce55240 con 0x5624ccf50ca0 2024-04-22T06:32:12.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.203+0000 7f2b2b5af740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5624cceb3ad0 con 0x5624ccf93e40 2024-04-22T06:32:12.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.203+0000 7f2b21d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624ccf54ec0 0x5624ccf9d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.203+0000 7f2b21d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624ccf54ec0 0x5624ccf9d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36548/0 (socket says 172.21.15.44:36548) 2024-04-22T06:32:12.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.203+0000 7f2b21d9b640 1 -- 172.21.15.44:0/1547308236 learned_addr learned my addr 172.21.15.44:0/1547308236 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:12.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.204+0000 7f2b2259c640 1 --2- 172.21.15.44:0/1547308236 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5624ccf93e40 0x5624ccf9cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.204+0000 7f2b22d9d640 1 --2- 172.21.15.44:0/1547308236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5624ccf50ca0 0x5624ccf78ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.204+0000 7f2b21d9b640 1 -- 172.21.15.44:0/1547308236 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5624ccf93e40 msgr2=0x5624ccf9cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.204+0000 7f2b21d9b640 1 --2- 172.21.15.44:0/1547308236 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5624ccf93e40 0x5624ccf9cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.204+0000 7f2b21d9b640 1 -- 172.21.15.44:0/1547308236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5624ccf50ca0 msgr2=0x5624ccf78ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.204+0000 7f2b21d9b640 1 --2- 172.21.15.44:0/1547308236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5624ccf50ca0 0x5624ccf78ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.204+0000 7f2b21d9b640 1 -- 172.21.15.44:0/1547308236 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624ccbca2d0 con 0x5624ccf54ec0 2024-04-22T06:32:12.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.204+0000 7f2b22d9d640 1 --2- 172.21.15.44:0/1547308236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5624ccf50ca0 0x5624ccf78ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:32:12.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.204+0000 7f2b21d9b640 1 --2- 172.21.15.44:0/1547308236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624ccf54ec0 0x5624ccf9d4a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2b18005bf0 tx=0x7f2b18010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce88e8bf42b0688d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:12.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.204+0000 7f2b03fff640 1 -- 172.21.15.44:0/1547308236 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1800d900 con 0x5624ccf54ec0 2024-04-22T06:32:12.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.204+0000 7f2b03fff640 1 -- 172.21.15.44:0/1547308236 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b1800daa0 con 0x5624ccf54ec0 2024-04-22T06:32:12.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.204+0000 7f2b03fff640 1 -- 172.21.15.44:0/1547308236 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1800de50 con 0x5624ccf54ec0 2024-04-22T06:32:12.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.205+0000 7f2b2b5af740 1 -- 172.21.15.44:0/1547308236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624ccf54ec0 msgr2=0x5624ccf9d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.205+0000 7f2b2b5af740 1 --2- 172.21.15.44:0/1547308236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624ccf54ec0 0x5624ccf9d4a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2b18005bf0 tx=0x7f2b18010000 comp rx=0 tx=0).stop 2024-04-22T06:32:12.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.205+0000 7f2b2b5af740 1 -- 172.21.15.44:0/1547308236 shutdown_connections 2024-04-22T06:32:12.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.205+0000 7f2b2b5af740 1 --2- 172.21.15.44:0/1547308236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5624ccf50ca0 0x5624ccf78ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.205+0000 7f2b2b5af740 1 --2- 172.21.15.44:0/1547308236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624ccf54ec0 0x5624ccf9d4a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.205+0000 7f2b2b5af740 1 --2- 172.21.15.44:0/1547308236 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5624ccf93e40 0x5624ccf9cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.205+0000 7f2b2b5af740 1 -- 172.21.15.44:0/1547308236 >> 172.21.15.44:0/1547308236 conn(0x5624ccf4f990 msgr2=0x5624ccf52290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:12.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.205+0000 7f2b2b5af740 1 -- 172.21.15.44:0/1547308236 shutdown_connections 2024-04-22T06:32:12.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.205+0000 7f2b2b5af740 1 -- 172.21.15.44:0/1547308236 wait complete. 2024-04-22T06:32:12.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.205+0000 7f2b2b5af740 1 -- start start 2024-04-22T06:32:12.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.205+0000 7f2b2b5af740 1 Processor -- start 2024-04-22T06:32:12.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.206+0000 7f2b2b5af740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5624ccf54ec0 0x5624ccfa5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.206+0000 7f2b2b5af740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5624ccf93e40 0x5624ccf95580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.206+0000 7f2b2b5af740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624ccf9be20 0x5624ccf8d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.206+0000 7f2b2b5af740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5624ccf53250 con 0x5624ccf9be20 2024-04-22T06:32:12.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.206+0000 7f2b2b5af740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5624cce40190 con 0x5624ccf93e40 2024-04-22T06:32:12.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.206+0000 7f2b2b5af740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5624ccf53530 con 0x5624ccf54ec0 2024-04-22T06:32:12.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.206+0000 7f2b2b5af740 10 client.0 _ll_register_callbacks cb 0x5624cce3b8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T06:32:12.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.206+0000 7f2b22d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624ccf9be20 0x5624ccf8d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.206+0000 7f2b22d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624ccf9be20 0x5624ccf8d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36556/0 (socket says 172.21.15.44:36556) 2024-04-22T06:32:12.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.206+0000 7f2b22d9d640 1 -- 172.21.15.44:0/1999291270 learned_addr learned my addr 172.21.15.44:0/1999291270 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:12.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.207+0000 7f2b2259c640 1 --2- 172.21.15.44:0/1999291270 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5624ccf54ec0 0x5624ccfa5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.207+0000 7f2b21d9b640 1 --2- 172.21.15.44:0/1999291270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5624ccf93e40 0x5624ccf95580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.207+0000 7f2b22d9d640 1 -- 172.21.15.44:0/1999291270 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5624ccf54ec0 msgr2=0x5624ccfa5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.207+0000 7f2b22d9d640 1 --2- 172.21.15.44:0/1999291270 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5624ccf54ec0 0x5624ccfa5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.207+0000 7f2b22d9d640 1 -- 172.21.15.44:0/1999291270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5624ccf93e40 msgr2=0x5624ccf95580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.207+0000 7f2b22d9d640 1 --2- 172.21.15.44:0/1999291270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5624ccf93e40 0x5624ccf95580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.207+0000 7f2b22d9d640 1 -- 172.21.15.44:0/1999291270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624ccc946b0 con 0x5624ccf9be20 2024-04-22T06:32:12.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.207+0000 7f2b22d9d640 1 --2- 172.21.15.44:0/1999291270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624ccf9be20 0x5624ccf8d710 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c037040 tx=0x7f2b1c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d1cef48b8d0ec36 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:12.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.207+0000 7f2b017fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T06:32:12.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.207+0000 7f2b017fa640 1 -- 172.21.15.44:0/1999291270 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c045070 con 0x5624ccf9be20 2024-04-22T06:32:12.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.207+0000 7f2b017fa640 1 -- 172.21.15.44:0/1999291270 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f2b1c0349f0 con 0x5624ccf9be20 2024-04-22T06:32:12.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.207+0000 7f2b2b5af740 1 -- 172.21.15.44:0/1999291270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b1800daa0 con 0x5624ccf9be20 2024-04-22T06:32:12.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.207+0000 7f2b017fa640 1 -- 172.21.15.44:0/1999291270 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f2b1c050050 con 0x5624ccf9be20 2024-04-22T06:32:12.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.207+0000 7f2b2b5af740 10 client.5190 fetch_fsmap learned FSMap version 18 2024-04-22T06:32:12.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.207+0000 7f2b2b5af740 1 -- 172.21.15.44:0/1999291270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x5624ccf96060 con 0x5624ccf9be20 2024-04-22T06:32:12.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.207+0000 7f2b017fa640 1 -- 172.21.15.44:0/1999291270 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b1800daa0 con 0x5624ccf9be20 2024-04-22T06:32:12.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.208+0000 7f2b017fa640 1 -- 172.21.15.44:0/1999291270 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 18) ==== 4792+0+0 (secure 0 0 0) 0x7f2b1c053020 con 0x5624ccf9be20 2024-04-22T06:32:12.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.208+0000 7f2b2b5af740 10 client.5190 fetch_fsmap finished waiting for FSMap version 18 2024-04-22T06:32:12.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.208+0000 7f2b2b5af740 10 client.5190 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-22T06:32:12.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.208+0000 7f2b2b5af740 20 client.5190 populate_metadata read hostname 'smithi044' 2024-04-22T06:32:12.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.208+0000 7f2b2b5af740 1 --2- 172.21.15.44:0/1999291270 >> [v2:172.21.15.44:6832/1497302315,v1:172.21.15.44:6834/1497302315] conn(0x5624ccfa8620 0x5624ccfb3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.208+0000 7f2b2b5af740 4 client.5190 mds_command: new command op to 4287 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T06:32:12.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.208+0000 7f2b2b5af740 1 -- 172.21.15.44:0/1999291270 --> [v2:172.21.15.44:6832/1497302315,v1:172.21.15.44:6834/1497302315] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624ccec5530 con 0x5624ccfa8620 2024-04-22T06:32:12.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.208+0000 7f2b2259c640 1 --2- 172.21.15.44:0/1999291270 >> [v2:172.21.15.44:6832/1497302315,v1:172.21.15.44:6834/1497302315] conn(0x5624ccfa8620 0x5624ccfb3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.208+0000 7f2b017fa640 10 client.5190 ms_handle_connect on v2:172.21.15.44:6832/1497302315 2024-04-22T06:32:12.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.208+0000 7f2b2259c640 1 --2- 172.21.15.44:0/1999291270 >> [v2:172.21.15.44:6832/1497302315,v1:172.21.15.44:6834/1497302315] conn(0x5624ccfa8620 0x5624ccfb3e30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=26ccf99d1afc297b server_cookie=dc1c097cf8f92766 in_seq=0 out_seq=0 2024-04-22T06:32:12.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.209+0000 7f2b017fa640 1 -- 172.21.15.44:0/1999291270 <== mds.4 v2:172.21.15.44:6832/1497302315 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5624ccec5530 con 0x5624ccfa8620 2024-04-22T06:32:12.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.209+0000 7f2b017fa640 10 client.5190 handle_command_reply: tid=0 2024-04-22T06:32:12.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.209+0000 7f2b2b5af740 1 client.5190 shutdown 2024-04-22T06:32:12.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.209+0000 7f2b2b5af740 1 -- 172.21.15.44:0/1999291270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624ccf9be20 msgr2=0x5624ccf8d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.209+0000 7f2b2b5af740 1 --2- 172.21.15.44:0/1999291270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624ccf9be20 0x5624ccf8d710 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c037040 tx=0x7f2b1c004ab0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.209+0000 7f2b2b5af740 1 -- 172.21.15.44:0/1999291270 shutdown_connections 2024-04-22T06:32:12.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.209+0000 7f2b2b5af740 1 --2- 172.21.15.44:0/1999291270 >> [v2:172.21.15.44:6832/1497302315,v1:172.21.15.44:6834/1497302315] conn(0x5624ccfa8620 0x5624ccfb3e30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.209+0000 7f2b2b5af740 1 --2- 172.21.15.44:0/1999291270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624ccf9be20 0x5624ccf8d710 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.209+0000 7f2b2b5af740 1 --2- 172.21.15.44:0/1999291270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5624ccf93e40 0x5624ccf95580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.209+0000 7f2b017fa640 0 client.5190 ms_handle_reset on v2:172.21.15.44:6832/1497302315 2024-04-22T06:32:12.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.209+0000 7f2b2b5af740 1 --2- 172.21.15.44:0/1999291270 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5624ccf54ec0 0x5624ccfa5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.209+0000 7f2b2b5af740 1 -- 172.21.15.44:0/1999291270 >> 172.21.15.44:0/1999291270 conn(0x5624ccf50ca0 msgr2=0x5624ccf94220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:12.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.209+0000 7f2b2b5af740 1 -- 172.21.15.44:0/1999291270 shutdown_connections 2024-04-22T06:32:12.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.209+0000 7f2b2b5af740 1 -- 172.21.15.44:0/1999291270 wait complete. 2024-04-22T06:32:12.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.209+0000 7f2b2b5af740 20 client.5190 trim_cache size 0 max 16384 2024-04-22T06:32:12.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.228+0000 7f2b2b5af740 1 Processor -- start 2024-04-22T06:32:12.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.228+0000 7f2b2b5af740 1 -- start start 2024-04-22T06:32:12.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.228+0000 7f2b2b5af740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5624ccf54ec0 0x5624ccf94220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.228+0000 7f2b2b5af740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5624ccf93e40 0x5624ccf94760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.228+0000 7f2b2b5af740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624ccf9be20 0x5624ccf8d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.228+0000 7f2b2b5af740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2b241fd0d0 con 0x5624ccf9be20 2024-04-22T06:32:12.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.228+0000 7f2b2b5af740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5624ccf6b8c0 con 0x5624ccf54ec0 2024-04-22T06:32:12.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.228+0000 7f2b2b5af740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5624ccf97ec0 con 0x5624ccf93e40 2024-04-22T06:32:12.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.229+0000 7f2b22d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624ccf9be20 0x5624ccf8d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.229+0000 7f2b22d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624ccf9be20 0x5624ccf8d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36562/0 (socket says 172.21.15.44:36562) 2024-04-22T06:32:12.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.229+0000 7f2b22d9d640 1 -- 172.21.15.44:0/3643066930 learned_addr learned my addr 172.21.15.44:0/3643066930 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:12.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.229+0000 7f2b2259c640 1 --2- 172.21.15.44:0/3643066930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5624ccf54ec0 0x5624ccf94220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.229+0000 7f2b21d9b640 1 --2- 172.21.15.44:0/3643066930 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5624ccf93e40 0x5624ccf94760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.229+0000 7f2b22d9d640 1 -- 172.21.15.44:0/3643066930 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5624ccf93e40 msgr2=0x5624ccf94760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.229+0000 7f2b22d9d640 1 --2- 172.21.15.44:0/3643066930 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5624ccf93e40 0x5624ccf94760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.229+0000 7f2b22d9d640 1 -- 172.21.15.44:0/3643066930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5624ccf54ec0 msgr2=0x5624ccf94220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.229+0000 7f2b22d9d640 1 --2- 172.21.15.44:0/3643066930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5624ccf54ec0 0x5624ccf94220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.229+0000 7f2b22d9d640 1 -- 172.21.15.44:0/3643066930 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624ccf50720 con 0x5624ccf9be20 2024-04-22T06:32:12.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.229+0000 7f2b22d9d640 1 --2- 172.21.15.44:0/3643066930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624ccf9be20 0x5624ccf8d070 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x5624ccf9c240 tx=0x7f2b1c03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35855c3936becf52 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:12.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.229+0000 7f2b037fe640 1 -- 172.21.15.44:0/3643066930 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c045040 con 0x5624ccf9be20 2024-04-22T06:32:12.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.229+0000 7f2b037fe640 1 -- 172.21.15.44:0/3643066930 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b1c034a10 con 0x5624ccf9be20 2024-04-22T06:32:12.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.229+0000 7f2b037fe640 1 -- 172.21.15.44:0/3643066930 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c039040 con 0x5624ccf9be20 2024-04-22T06:32:12.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.230+0000 7f2b2b5af740 1 -- 172.21.15.44:0/3643066930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624ccf9be20 msgr2=0x5624ccf8d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.230+0000 7f2b2b5af740 1 --2- 172.21.15.44:0/3643066930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624ccf9be20 0x5624ccf8d070 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x5624ccf9c240 tx=0x7f2b1c03f000 comp rx=0 tx=0).stop 2024-04-22T06:32:12.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.230+0000 7f2b2b5af740 1 -- 172.21.15.44:0/3643066930 shutdown_connections 2024-04-22T06:32:12.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.230+0000 7f2b2b5af740 1 --2- 172.21.15.44:0/3643066930 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5624ccf93e40 0x5624ccf94760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.230+0000 7f2b2b5af740 1 --2- 172.21.15.44:0/3643066930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624ccf9be20 0x5624ccf8d070 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.230+0000 7f2b2b5af740 1 --2- 172.21.15.44:0/3643066930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5624ccf54ec0 0x5624ccf94220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.230+0000 7f2b2b5af740 1 -- 172.21.15.44:0/3643066930 >> 172.21.15.44:0/3643066930 conn(0x5624ccf50ca0 msgr2=0x5624ccf95390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:12.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.230+0000 7f2b2b5af740 1 -- 172.21.15.44:0/3643066930 shutdown_connections 2024-04-22T06:32:12.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.230+0000 7f2b2b5af740 1 -- 172.21.15.44:0/3643066930 wait complete. 2024-04-22T06:32:12.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.230+0000 7f2b2b5af740 1 -- start start 2024-04-22T06:32:12.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.230+0000 7f2b2b5af740 1 Processor -- start 2024-04-22T06:32:12.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.231+0000 7f2b2b5af740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5624ccf54ec0 0x5624cd00fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.231+0000 7f2b2b5af740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624ccf93e40 0x5624cd009350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.231+0000 7f2b2b5af740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5624ccf9be20 0x5624ccf8c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.231+0000 7f2b2b5af740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5624ccfa84a0 con 0x5624ccf93e40 2024-04-22T06:32:12.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.231+0000 7f2b2b5af740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5624ccf8df60 con 0x5624ccf54ec0 2024-04-22T06:32:12.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.231+0000 7f2b2b5af740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5624ccfa8af0 con 0x5624ccf9be20 2024-04-22T06:32:12.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.231+0000 7f2b2b5af740 10 client.0 _ll_register_callbacks cb 0x7f2b1c037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T06:32:12.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.231+0000 7f2b21d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624ccf93e40 0x5624cd009350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.231+0000 7f2b21d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624ccf93e40 0x5624cd009350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36566/0 (socket says 172.21.15.44:36566) 2024-04-22T06:32:12.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.231+0000 7f2b21d9b640 1 -- 172.21.15.44:0/3002824825 learned_addr learned my addr 172.21.15.44:0/3002824825 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:12.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.232+0000 7f2b22d9d640 1 --2- 172.21.15.44:0/3002824825 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5624ccf9be20 0x5624ccf8c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.232+0000 7f2b2259c640 1 --2- 172.21.15.44:0/3002824825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5624ccf54ec0 0x5624cd00fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.232+0000 7f2b21d9b640 1 -- 172.21.15.44:0/3002824825 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5624ccf9be20 msgr2=0x5624ccf8c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.232+0000 7f2b21d9b640 1 --2- 172.21.15.44:0/3002824825 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5624ccf9be20 0x5624ccf8c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.232+0000 7f2b21d9b640 1 -- 172.21.15.44:0/3002824825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5624ccf54ec0 msgr2=0x5624cd00fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.232+0000 7f2b21d9b640 1 --2- 172.21.15.44:0/3002824825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5624ccf54ec0 0x5624cd00fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.232+0000 7f2b21d9b640 1 -- 172.21.15.44:0/3002824825 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624cd0125a0 con 0x5624ccf93e40 2024-04-22T06:32:12.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.232+0000 7f2b22d9d640 1 --2- 172.21.15.44:0/3002824825 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5624ccf9be20 0x5624ccf8c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:32:12.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.232+0000 7f2b2259c640 1 --2- 172.21.15.44:0/3002824825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5624ccf54ec0 0x5624cd00fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:32:12.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.232+0000 7f2b21d9b640 1 --2- 172.21.15.44:0/3002824825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624ccf93e40 0x5624cd009350 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2b18005b90 tx=0x7f2b18012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fdc379b861f257 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:12.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.232+0000 7f2b017fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T06:32:12.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.232+0000 7f2b017fa640 1 -- 172.21.15.44:0/3002824825 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b18020ba0 con 0x5624ccf93e40 2024-04-22T06:32:12.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.232+0000 7f2b017fa640 1 -- 172.21.15.44:0/3002824825 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f2b18020d40 con 0x5624ccf93e40 2024-04-22T06:32:12.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.232+0000 7f2b2b5af740 1 -- 172.21.15.44:0/3002824825 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b1c034a10 con 0x5624ccf93e40 2024-04-22T06:32:12.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.232+0000 7f2b017fa640 1 -- 172.21.15.44:0/3002824825 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f2b18036050 con 0x5624ccf93e40 2024-04-22T06:32:12.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.232+0000 7f2b2b5af740 10 client.5196 fetch_fsmap learned FSMap version 18 2024-04-22T06:32:12.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.232+0000 7f2b2b5af740 1 -- 172.21.15.44:0/3002824825 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x5624cd00f3c0 con 0x5624ccf93e40 2024-04-22T06:32:12.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.232+0000 7f2b017fa640 1 -- 172.21.15.44:0/3002824825 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b1c034a10 con 0x5624ccf93e40 2024-04-22T06:32:12.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.232+0000 7f2b017fa640 1 -- 172.21.15.44:0/3002824825 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 18) ==== 4792+0+0 (secure 0 0 0) 0x7f2b18039020 con 0x5624ccf93e40 2024-04-22T06:32:12.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.232+0000 7f2b2b5af740 10 client.5196 fetch_fsmap finished waiting for FSMap version 18 2024-04-22T06:32:12.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.233+0000 7f2b2b5af740 10 client.5196 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-22T06:32:12.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.233+0000 7f2b2b5af740 20 client.5196 populate_metadata read hostname 'smithi044' 2024-04-22T06:32:12.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.233+0000 7f2b2b5af740 1 --2- 172.21.15.44:0/3002824825 >> [v2:172.21.15.44:6832/1497302315,v1:172.21.15.44:6834/1497302315] conn(0x5624ccf77650 0x5624ccf77a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.233+0000 7f2b2b5af740 4 client.5196 mds_command: new command op to 4287 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T06:32:12.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.233+0000 7f2b2b5af740 1 -- 172.21.15.44:0/3002824825 --> [v2:172.21.15.44:6832/1497302315,v1:172.21.15.44:6834/1497302315] -- command(tid 0: {"prefix": "status"}) -- 0x5624ccebf3a0 con 0x5624ccf77650 2024-04-22T06:32:12.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.233+0000 7f2b2259c640 1 --2- 172.21.15.44:0/3002824825 >> [v2:172.21.15.44:6832/1497302315,v1:172.21.15.44:6834/1497302315] conn(0x5624ccf77650 0x5624ccf77a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.233+0000 7f2b2259c640 1 --2- 172.21.15.44:0/3002824825 >> [v2:172.21.15.44:6832/1497302315,v1:172.21.15.44:6834/1497302315] conn(0x5624ccf77650 0x5624ccf77a30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=28b9279584c79ec9 server_cookie=1c6647a48c3a41d5 in_seq=0 out_seq=0 2024-04-22T06:32:12.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.233+0000 7f2b017fa640 10 client.5196 ms_handle_connect on v2:172.21.15.44:6832/1497302315 2024-04-22T06:32:12.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.233+0000 7f2b017fa640 1 -- 172.21.15.44:0/3002824825 <== mds.4 v2:172.21.15.44:6832/1497302315 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5624ccebf3a0 con 0x5624ccf77650 2024-04-22T06:32:12.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.233+0000 7f2b017fa640 10 client.5196 handle_command_reply: tid=0 2024-04-22T06:32:12.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.233+0000 7f2b2b5af740 1 client.5196 shutdown 2024-04-22T06:32:12.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.233+0000 7f2b2b5af740 1 -- 172.21.15.44:0/3002824825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624ccf93e40 msgr2=0x5624cd009350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.233+0000 7f2b2b5af740 1 --2- 172.21.15.44:0/3002824825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624ccf93e40 0x5624cd009350 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2b18005b90 tx=0x7f2b18012000 comp rx=0 tx=0).stop 2024-04-22T06:32:12.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.233+0000 7f2b2b5af740 1 -- 172.21.15.44:0/3002824825 shutdown_connections 2024-04-22T06:32:12.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.233+0000 7f2b2b5af740 1 --2- 172.21.15.44:0/3002824825 >> [v2:172.21.15.44:6832/1497302315,v1:172.21.15.44:6834/1497302315] conn(0x5624ccf77650 0x5624ccf77a30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.233+0000 7f2b2b5af740 1 --2- 172.21.15.44:0/3002824825 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5624ccf9be20 0x5624ccf8c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.233+0000 7f2b2b5af740 1 --2- 172.21.15.44:0/3002824825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5624ccf93e40 0x5624cd009350 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.233+0000 7f2b017fa640 0 client.5196 ms_handle_reset on v2:172.21.15.44:6832/1497302315 2024-04-22T06:32:12.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.233+0000 7f2b2b5af740 1 --2- 172.21.15.44:0/3002824825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5624ccf54ec0 0x5624cd00fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.233+0000 7f2b2b5af740 1 -- 172.21.15.44:0/3002824825 >> 172.21.15.44:0/3002824825 conn(0x5624ccf50ca0 msgr2=0x5624ccf4e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:12.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.234+0000 7f2b2b5af740 1 -- 172.21.15.44:0/3002824825 shutdown_connections 2024-04-22T06:32:12.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.234+0000 7f2b2b5af740 1 -- 172.21.15.44:0/3002824825 wait complete. 2024-04-22T06:32:12.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.234+0000 7f2b2b5af740 20 client.5196 trim_cache size 0 max 16384 2024-04-22T06:32:12.240 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-22T06:32:12.240 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "9b0a34ca-403d-4513-8a66-d3749ba559aa", 2024-04-22T06:32:12.240 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 4, 2024-04-22T06:32:12.240 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 4287, 2024-04-22T06:32:12.240 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-04-22T06:32:12.240 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-04-22T06:32:12.240 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-04-22T06:32:12.240 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 22.081698198000002, 2024-04-22T06:32:12.240 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 18, 2024-04-22T06:32:12.240 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 27, 2024-04-22T06:32:12.240 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 27, 2024-04-22T06:32:12.240 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 72.142445487000003 2024-04-22T06:32:12.241 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-22T06:32:12.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.234+0000 7f2b288e0640 1 -- 172.21.15.44:0/4128731766 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f2b0406c8f0 msgr2=0x7f2b0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.234+0000 7f2b288e0640 1 --2- 172.21.15.44:0/4128731766 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f2b0406c8f0 0x7f2b0406edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2b2414a5a0 tx=0x7f2b18009040 comp rx=0 tx=0).stop 2024-04-22T06:32:12.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.234+0000 7f2b288e0640 1 -- 172.21.15.44:0/4128731766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b240c5c00 msgr2=0x7f2b240cfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.234+0000 7f2b288e0640 1 --2- 172.21.15.44:0/4128731766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b240c5c00 0x7f2b240cfea0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2b1400c400 tx=0x7f2b1400c8c0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.234+0000 7f2b288e0640 1 -- 172.21.15.44:0/4128731766 shutdown_connections 2024-04-22T06:32:12.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.234+0000 7f2b288e0640 1 --2- 172.21.15.44:0/4128731766 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f2b0406c8f0 0x7f2b0406edb0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.234+0000 7f2b288e0640 1 --2- 172.21.15.44:0/4128731766 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b24155460 0x7f2b241f1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.234+0000 7f2b288e0640 1 --2- 172.21.15.44:0/4128731766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b2415f050 0x7f2b240cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.234+0000 7f2b288e0640 1 --2- 172.21.15.44:0/4128731766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b240c5c00 0x7f2b240cfea0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.234+0000 7f2b288e0640 1 -- 172.21.15.44:0/4128731766 >> 172.21.15.44:0/4128731766 conn(0x7f2b2406cca0 msgr2=0x7f2b2414a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:12.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.234+0000 7f2b288e0640 1 -- 172.21.15.44:0/4128731766 shutdown_connections 2024-04-22T06:32:12.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.234+0000 7f2b288e0640 1 -- 172.21.15.44:0/4128731766 wait complete. 2024-04-22T06:32:12.244 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-22T06:32:12.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.346+0000 7f32e8cd2640 1 Processor -- start 2024-04-22T06:32:12.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.346+0000 7f32e8cd2640 1 -- start start 2024-04-22T06:32:12.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.347+0000 7f32e8cd2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f32e4154f00 0x7f32e40cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.347+0000 7f32e8cd2640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32e41552e0 0x7f32e415c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.347+0000 7f32e8cd2640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f32e406d470 0x7f32e40d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.347+0000 7f32e8cd2640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f32e40c6740 con 0x7f32e406d470 2024-04-22T06:32:12.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.347+0000 7f32e8cd2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f32e40c5b30 con 0x7f32e4154f00 2024-04-22T06:32:12.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.347+0000 7f32e8cd2640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f32e40c5e10 con 0x7f32e41552e0 2024-04-22T06:32:12.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.348+0000 7f32e2d9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f32e406d470 0x7f32e40d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.348+0000 7f32e2d9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f32e406d470 0x7f32e40d0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36582/0 (socket says 172.21.15.44:36582) 2024-04-22T06:32:12.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.348+0000 7f32e2d9d640 1 -- 172.21.15.44:0/240322874 learned_addr learned my addr 172.21.15.44:0/240322874 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:12.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.348+0000 7f32e1d9b640 1 --2- 172.21.15.44:0/240322874 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32e41552e0 0x7f32e415c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.348+0000 7f32e259c640 1 --2- 172.21.15.44:0/240322874 >> v2:172.21.15.70:3300/0 conn(0x7f32e4154f00 0x7f32e40cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.348+0000 7f32e2d9d640 1 -- 172.21.15.44:0/240322874 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32e41552e0 msgr2=0x7f32e415c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.348+0000 7f32e2d9d640 1 --2- 172.21.15.44:0/240322874 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32e41552e0 0x7f32e415c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.348+0000 7f32e2d9d640 1 -- 172.21.15.44:0/240322874 >> v2:172.21.15.70:3300/0 conn(0x7f32e4154f00 msgr2=0x7f32e40cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.348+0000 7f32e2d9d640 1 --2- 172.21.15.44:0/240322874 >> v2:172.21.15.70:3300/0 conn(0x7f32e4154f00 0x7f32e40cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.348+0000 7f32e2d9d640 1 -- 172.21.15.44:0/240322874 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e40c60f0 con 0x7f32e406d470 2024-04-22T06:32:12.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.349+0000 7f32e2d9d640 1 --2- 172.21.15.44:0/240322874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32e406d470 0x7f32e40d0f80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f32e406cb60 tx=0x7f32dc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd4af9f815b1975 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:12.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.349+0000 7f32e159a640 1 -- 172.21.15.44:0/240322874 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc035020 con 0x7f32e406d470 2024-04-22T06:32:12.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.349+0000 7f32e159a640 1 -- 172.21.15.44:0/240322874 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32dc031870 con 0x7f32e406d470 2024-04-22T06:32:12.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.349+0000 7f32e159a640 1 -- 172.21.15.44:0/240322874 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc031c40 con 0x7f32e406d470 2024-04-22T06:32:12.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.349+0000 7f32e8cd2640 1 -- 172.21.15.44:0/240322874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32e406d470 msgr2=0x7f32e40d0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.349+0000 7f32e8cd2640 1 --2- 172.21.15.44:0/240322874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32e406d470 0x7f32e40d0f80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f32e406cb60 tx=0x7f32dc02b910 comp rx=0 tx=0).stop 2024-04-22T06:32:12.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.349+0000 7f32e8cd2640 1 -- 172.21.15.44:0/240322874 shutdown_connections 2024-04-22T06:32:12.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.349+0000 7f32e8cd2640 1 --2- 172.21.15.44:0/240322874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32e406d470 0x7f32e40d0f80 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.349+0000 7f32e8cd2640 1 --2- 172.21.15.44:0/240322874 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32e41552e0 0x7f32e415c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.349+0000 7f32e8cd2640 1 --2- 172.21.15.44:0/240322874 >> v2:172.21.15.70:3300/0 conn(0x7f32e4154f00 0x7f32e40cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.349+0000 7f32e8cd2640 1 -- 172.21.15.44:0/240322874 >> 172.21.15.44:0/240322874 conn(0x7f32e40733c0 msgr2=0x7f32e40737d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:12.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.349+0000 7f32e8cd2640 1 -- 172.21.15.44:0/240322874 shutdown_connections 2024-04-22T06:32:12.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.349+0000 7f32e8cd2640 1 -- 172.21.15.44:0/240322874 wait complete. 2024-04-22T06:32:12.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.351+0000 7f32e8cd2640 1 Processor -- start 2024-04-22T06:32:12.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.351+0000 7f32e8cd2640 1 -- start start 2024-04-22T06:32:12.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.351+0000 7f32e8cd2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32e406d470 0x7f32e40cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.351+0000 7f32e8cd2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32e4154f00 0x7f32e41f3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.351+0000 7f32e8cd2640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32e41552e0 0x7f32e40d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.351+0000 7f32e8cd2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f32e40c6d50 con 0x7f32e406d470 2024-04-22T06:32:12.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.351+0000 7f32e8cd2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f32e40c6bd0 con 0x7f32e4154f00 2024-04-22T06:32:12.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.351+0000 7f32e8cd2640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f32e415cb20 con 0x7f32e41552e0 2024-04-22T06:32:12.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.352+0000 7f32e259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32e406d470 0x7f32e40cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.352+0000 7f32e259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32e406d470 0x7f32e40cfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36588/0 (socket says 172.21.15.44:36588) 2024-04-22T06:32:12.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.352+0000 7f32e259c640 1 -- 172.21.15.44:0/1753929760 learned_addr learned my addr 172.21.15.44:0/1753929760 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:12.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.352+0000 7f32e1d9b640 1 --2- 172.21.15.44:0/1753929760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32e4154f00 0x7f32e41f3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.352+0000 7f32e2d9d640 1 --2- 172.21.15.44:0/1753929760 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32e41552e0 0x7f32e40d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.352+0000 7f32e259c640 1 -- 172.21.15.44:0/1753929760 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32e41552e0 msgr2=0x7f32e40d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.352+0000 7f32e259c640 1 --2- 172.21.15.44:0/1753929760 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32e41552e0 0x7f32e40d0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.352+0000 7f32e259c640 1 -- 172.21.15.44:0/1753929760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32e4154f00 msgr2=0x7f32e41f3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.352+0000 7f32e259c640 1 --2- 172.21.15.44:0/1753929760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32e4154f00 0x7f32e41f3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.352+0000 7f32e259c640 1 -- 172.21.15.44:0/1753929760 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e40d0530 con 0x7f32e406d470 2024-04-22T06:32:12.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.352+0000 7f32e1d9b640 1 --2- 172.21.15.44:0/1753929760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32e4154f00 0x7f32e41f3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:32:12.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.352+0000 7f32e259c640 1 --2- 172.21.15.44:0/1753929760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32e406d470 0x7f32e40cfdb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f32d8007e60 tx=0x7f32d800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:12.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.352+0000 7f32d2ffd640 1 -- 172.21.15.44:0/1753929760 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d8014620 con 0x7f32e406d470 2024-04-22T06:32:12.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.352+0000 7f32d2ffd640 1 -- 172.21.15.44:0/1753929760 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32d8019070 con 0x7f32e406d470 2024-04-22T06:32:12.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.352+0000 7f32e8cd2640 1 -- 172.21.15.44:0/1753929760 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32e41f0700 con 0x7f32e406d470 2024-04-22T06:32:12.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.352+0000 7f32d2ffd640 1 -- 172.21.15.44:0/1753929760 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d8014a00 con 0x7f32e406d470 2024-04-22T06:32:12.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.352+0000 7f32e8cd2640 1 -- 172.21.15.44:0/1753929760 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32e41f0cc0 con 0x7f32e406d470 2024-04-22T06:32:12.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.354+0000 7f32d2ffd640 1 -- 172.21.15.44:0/1753929760 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f32d8014ba0 con 0x7f32e406d470 2024-04-22T06:32:12.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.354+0000 7f32d2ffd640 1 --2- 172.21.15.44:0/1753929760 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f32c406c8f0 0x7f32c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.354+0000 7f32d2ffd640 1 -- 172.21.15.44:0/1753929760 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f32d808c8a0 con 0x7f32e406d470 2024-04-22T06:32:12.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.355+0000 7f32e1d9b640 1 --2- 172.21.15.44:0/1753929760 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f32c406c8f0 0x7f32c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.355+0000 7f32e1d9b640 1 --2- 172.21.15.44:0/1753929760 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f32c406c8f0 0x7f32c406edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f32d4006fd0 tx=0x7f32d4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:12.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.355+0000 7f32eb99e740 1 Processor -- start 2024-04-22T06:32:12.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.355+0000 7f32eb99e740 1 -- start start 2024-04-22T06:32:12.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.356+0000 7f32eb99e740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b74bb10e40 0x55b74bb19f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.356+0000 7f32eb99e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74bad1ec0 0x55b74bb1a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.356+0000 7f32eb99e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74bacdca0 0x55b74baf5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.356+0000 7f32eb99e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b74ba34b10 con 0x55b74bacdca0 2024-04-22T06:32:12.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.356+0000 7f32eb99e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55b74b9d2240 con 0x55b74bad1ec0 2024-04-22T06:32:12.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.356+0000 7f32eb99e740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55b74ba30ad0 con 0x55b74bb10e40 2024-04-22T06:32:12.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.356+0000 7f32e2d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74bacdca0 0x55b74baf5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.356+0000 7f32e1d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74bad1ec0 0x55b74bb1a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.356+0000 7f32e259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b74bb10e40 0x55b74bb19f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.356+0000 7f32e2d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74bacdca0 0x55b74baf5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36600/0 (socket says 172.21.15.44:36600) 2024-04-22T06:32:12.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.356+0000 7f32e259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b74bb10e40 0x55b74bb19f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.44:35950/0 (socket says 172.21.15.44:35950) 2024-04-22T06:32:12.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.356+0000 7f32e2d9d640 1 -- 172.21.15.44:0/747159851 learned_addr learned my addr 172.21.15.44:0/747159851 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:12.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.356+0000 7f32e1d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74bad1ec0 0x55b74bb1a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:52822/0 (socket says 172.21.15.44:52822) 2024-04-22T06:32:12.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.356+0000 7f32e2d9d640 1 -- 172.21.15.44:0/747159851 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b74bb10e40 msgr2=0x55b74bb19f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.356+0000 7f32e2d9d640 1 --2- 172.21.15.44:0/747159851 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b74bb10e40 0x55b74bb19f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.357+0000 7f32e2d9d640 1 -- 172.21.15.44:0/747159851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74bad1ec0 msgr2=0x55b74bb1a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.357+0000 7f32e2d9d640 1 --2- 172.21.15.44:0/747159851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74bad1ec0 0x55b74bb1a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.357+0000 7f32e2d9d640 1 -- 172.21.15.44:0/747159851 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b74b7472d0 con 0x55b74bacdca0 2024-04-22T06:32:12.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.357+0000 7f32e1d9b640 1 --2- 172.21.15.44:0/747159851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74bad1ec0 0x55b74bb1a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:32:12.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.357+0000 7f32e2d9d640 1 --2- 172.21.15.44:0/747159851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74bacdca0 0x55b74baf5ee0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f32dc02b950 tx=0x7f32dc030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c921773c43b38a24 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:12.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.357+0000 7f32c3fff640 1 -- 172.21.15.44:0/747159851 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc035040 con 0x55b74bacdca0 2024-04-22T06:32:12.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.357+0000 7f32c3fff640 1 -- 172.21.15.44:0/747159851 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32dc031970 con 0x55b74bacdca0 2024-04-22T06:32:12.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.357+0000 7f32c3fff640 1 -- 172.21.15.44:0/747159851 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc031d60 con 0x55b74bacdca0 2024-04-22T06:32:12.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.357+0000 7f32eb99e740 1 -- 172.21.15.44:0/747159851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74bacdca0 msgr2=0x55b74baf5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.357+0000 7f32eb99e740 1 --2- 172.21.15.44:0/747159851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74bacdca0 0x55b74baf5ee0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f32dc02b950 tx=0x7f32dc030000 comp rx=0 tx=0).stop 2024-04-22T06:32:12.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.358+0000 7f32eb99e740 1 -- 172.21.15.44:0/747159851 shutdown_connections 2024-04-22T06:32:12.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.358+0000 7f32eb99e740 1 --2- 172.21.15.44:0/747159851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74bacdca0 0x55b74baf5ee0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.358+0000 7f32eb99e740 1 --2- 172.21.15.44:0/747159851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74bad1ec0 0x55b74bb1a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.358+0000 7f32eb99e740 1 --2- 172.21.15.44:0/747159851 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b74bb10e40 0x55b74bb19f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.358+0000 7f32eb99e740 1 -- 172.21.15.44:0/747159851 >> 172.21.15.44:0/747159851 conn(0x55b74bacc990 msgr2=0x55b74bacf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:12.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.358+0000 7f32eb99e740 1 -- 172.21.15.44:0/747159851 shutdown_connections 2024-04-22T06:32:12.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.358+0000 7f32eb99e740 1 -- 172.21.15.44:0/747159851 wait complete. 2024-04-22T06:32:12.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.358+0000 7f32eb99e740 1 -- start start 2024-04-22T06:32:12.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.358+0000 7f32eb99e740 1 Processor -- start 2024-04-22T06:32:12.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.359+0000 7f32eb99e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74bad1ec0 0x55b74bb16f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.359+0000 7f32e259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74bad1ec0 0x55b74bb16f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.359+0000 7f32e259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74bad1ec0 0x55b74bb16f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36614/0 (socket says 172.21.15.44:36614) 2024-04-22T06:32:12.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.359+0000 7f32eb99e740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b74bb10e40 0x55b74baf61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.359+0000 7f32e259c640 1 -- 172.21.15.44:0/1410947244 learned_addr learned my addr 172.21.15.44:0/1410947244 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:12.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.359+0000 7f32eb99e740 1 --2- 172.21.15.44:0/1410947244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74bb16290 0x55b74bae3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.359+0000 7f32eb99e740 1 -- 172.21.15.44:0/1410947244 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b74bad0250 con 0x55b74bad1ec0 2024-04-22T06:32:12.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.359+0000 7f32eb99e740 1 -- 172.21.15.44:0/1410947244 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55b74b9bd190 con 0x55b74bb16290 2024-04-22T06:32:12.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.359+0000 7f32eb99e740 1 -- 172.21.15.44:0/1410947244 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55b74bad0530 con 0x55b74bb10e40 2024-04-22T06:32:12.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.359+0000 7f32eb99e740 10 client.0 _ll_register_callbacks cb 0x55b74b9b88b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T06:32:12.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.359+0000 7f32e2d9d640 1 --2- 172.21.15.44:0/1410947244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74bb16290 0x55b74bae3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.360+0000 7f32e1d9b640 1 --2- 172.21.15.44:0/1410947244 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b74bb10e40 0x55b74baf61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.360+0000 7f32e259c640 1 -- 172.21.15.44:0/1410947244 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b74bb10e40 msgr2=0x55b74baf61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.360+0000 7f32e259c640 1 --2- 172.21.15.44:0/1410947244 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b74bb10e40 0x55b74baf61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.360+0000 7f32e259c640 1 -- 172.21.15.44:0/1410947244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74bb16290 msgr2=0x55b74bae3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.360+0000 7f32e259c640 1 --2- 172.21.15.44:0/1410947244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74bb16290 0x55b74bae3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.360+0000 7f32e259c640 1 -- 172.21.15.44:0/1410947244 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b74b8116b0 con 0x55b74bad1ec0 2024-04-22T06:32:12.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.360+0000 7f32e2d9d640 1 --2- 172.21.15.44:0/1410947244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74bb16290 0x55b74bae3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:32:12.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.360+0000 7f32e1d9b640 1 --2- 172.21.15.44:0/1410947244 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b74bb10e40 0x55b74baf61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:32:12.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.360+0000 7f32e259c640 1 --2- 172.21.15.44:0/1410947244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74bad1ec0 0x55b74bb16f80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f32d805a0d0 tx=0x7f32d805a280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d4b7b0a7806d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:12.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.360+0000 7f32c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T06:32:12.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.360+0000 7f32c17fa640 1 -- 172.21.15.44:0/1410947244 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d805e170 con 0x55b74bad1ec0 2024-04-22T06:32:12.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.360+0000 7f32c17fa640 1 -- 172.21.15.44:0/1410947244 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f32d80567b0 con 0x55b74bad1ec0 2024-04-22T06:32:12.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.360+0000 7f32eb99e740 1 -- 172.21.15.44:0/1410947244 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32dc031970 con 0x55b74bad1ec0 2024-04-22T06:32:12.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.360+0000 7f32c17fa640 1 -- 172.21.15.44:0/1410947244 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f32d806ebe0 con 0x55b74bad1ec0 2024-04-22T06:32:12.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.360+0000 7f32eb99e740 10 client.5208 fetch_fsmap learned FSMap version 18 2024-04-22T06:32:12.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.360+0000 7f32eb99e740 1 -- 172.21.15.44:0/1410947244 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55b74bb157b0 con 0x55b74bad1ec0 2024-04-22T06:32:12.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.360+0000 7f32c17fa640 1 -- 172.21.15.44:0/1410947244 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32dc031970 con 0x55b74bad1ec0 2024-04-22T06:32:12.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.360+0000 7f32c17fa640 1 -- 172.21.15.44:0/1410947244 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 18) ==== 4792+0+0 (secure 0 0 0) 0x7f32d8079600 con 0x55b74bad1ec0 2024-04-22T06:32:12.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.360+0000 7f32eb99e740 10 client.5208 fetch_fsmap finished waiting for FSMap version 18 2024-04-22T06:32:12.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.360+0000 7f32eb99e740 10 client.5208 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-22T06:32:12.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.360+0000 7f32eb99e740 20 client.5208 populate_metadata read hostname 'smithi044' 2024-04-22T06:32:12.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.360+0000 7f32eb99e740 1 --2- 172.21.15.44:0/1410947244 >> [v2:172.21.15.44:6833/2383621014,v1:172.21.15.44:6835/2383621014] conn(0x55b74bb2e8a0 0x55b74bb30c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.360+0000 7f32eb99e740 4 client.5208 mds_command: new command op to 4290 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T06:32:12.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.360+0000 7f32eb99e740 1 -- 172.21.15.44:0/1410947244 --> [v2:172.21.15.44:6833/2383621014,v1:172.21.15.44:6835/2383621014] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b74ba42530 con 0x55b74bb2e8a0 2024-04-22T06:32:12.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.361+0000 7f32e1d9b640 1 --2- 172.21.15.44:0/1410947244 >> [v2:172.21.15.44:6833/2383621014,v1:172.21.15.44:6835/2383621014] conn(0x55b74bb2e8a0 0x55b74bb30c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.361+0000 7f32e1d9b640 1 --2- 172.21.15.44:0/1410947244 >> [v2:172.21.15.44:6833/2383621014,v1:172.21.15.44:6835/2383621014] conn(0x55b74bb2e8a0 0x55b74bb30c90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1becbc51b6ccc400 server_cookie=c59b9f464180376b in_seq=0 out_seq=0 2024-04-22T06:32:12.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.361+0000 7f32c17fa640 10 client.5208 ms_handle_connect on v2:172.21.15.44:6833/2383621014 2024-04-22T06:32:12.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.361+0000 7f32c17fa640 1 -- 172.21.15.44:0/1410947244 <== mds.3 v2:172.21.15.44:6833/2383621014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b74ba42530 con 0x55b74bb2e8a0 2024-04-22T06:32:12.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.361+0000 7f32c17fa640 10 client.5208 handle_command_reply: tid=0 2024-04-22T06:32:12.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.361+0000 7f32eb99e740 1 client.5208 shutdown 2024-04-22T06:32:12.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.362+0000 7f32eb99e740 1 -- 172.21.15.44:0/1410947244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74bad1ec0 msgr2=0x55b74bb16f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.362+0000 7f32eb99e740 1 --2- 172.21.15.44:0/1410947244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74bad1ec0 0x55b74bb16f80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f32d805a0d0 tx=0x7f32d805a280 comp rx=0 tx=0).stop 2024-04-22T06:32:12.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.362+0000 7f32eb99e740 1 -- 172.21.15.44:0/1410947244 shutdown_connections 2024-04-22T06:32:12.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.362+0000 7f32eb99e740 1 --2- 172.21.15.44:0/1410947244 >> [v2:172.21.15.44:6833/2383621014,v1:172.21.15.44:6835/2383621014] conn(0x55b74bb2e8a0 0x55b74bb30c90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.362+0000 7f32eb99e740 1 --2- 172.21.15.44:0/1410947244 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b74bb10e40 0x55b74baf61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.362+0000 7f32eb99e740 1 --2- 172.21.15.44:0/1410947244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74bb16290 0x55b74bae3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.362+0000 7f32eb99e740 1 --2- 172.21.15.44:0/1410947244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74bad1ec0 0x55b74bb16f80 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.362+0000 7f32c17fa640 0 client.5208 ms_handle_reset on v2:172.21.15.44:6833/2383621014 2024-04-22T06:32:12.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.362+0000 7f32eb99e740 1 -- 172.21.15.44:0/1410947244 >> 172.21.15.44:0/1410947244 conn(0x55b74bacdca0 msgr2=0x55b74bad0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:12.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.362+0000 7f32eb99e740 1 -- 172.21.15.44:0/1410947244 shutdown_connections 2024-04-22T06:32:12.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.362+0000 7f32eb99e740 1 -- 172.21.15.44:0/1410947244 wait complete. 2024-04-22T06:32:12.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.362+0000 7f32eb99e740 20 client.5208 trim_cache size 0 max 16384 2024-04-22T06:32:12.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.380+0000 7f32eb99e740 1 Processor -- start 2024-04-22T06:32:12.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.380+0000 7f32eb99e740 1 -- start start 2024-04-22T06:32:12.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.381+0000 7f32eb99e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74bad1ec0 0x55b74bb36530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.381+0000 7f32eb99e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74bb10e40 0x55b74bacf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.381+0000 7f32eb99e740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b74bb16290 0x55b74bacf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.381+0000 7f32eb99e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f32e41fefb0 con 0x55b74bb10e40 2024-04-22T06:32:12.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.381+0000 7f32eb99e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f32e415f310 con 0x55b74bad1ec0 2024-04-22T06:32:12.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.381+0000 7f32eb99e740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55b74baf6d00 con 0x55b74bb16290 2024-04-22T06:32:12.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.381+0000 7f32e1d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74bb10e40 0x55b74bacf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.381+0000 7f32e1d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74bb10e40 0x55b74bacf250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36616/0 (socket says 172.21.15.44:36616) 2024-04-22T06:32:12.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.381+0000 7f32e1d9b640 1 -- 172.21.15.44:0/2131423657 learned_addr learned my addr 172.21.15.44:0/2131423657 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:12.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.381+0000 7f32e2d9d640 1 --2- 172.21.15.44:0/2131423657 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b74bb16290 0x55b74bacf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.381+0000 7f32e259c640 1 --2- 172.21.15.44:0/2131423657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74bad1ec0 0x55b74bb36530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.381+0000 7f32e1d9b640 1 -- 172.21.15.44:0/2131423657 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b74bb16290 msgr2=0x55b74bacf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.381+0000 7f32e1d9b640 1 --2- 172.21.15.44:0/2131423657 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b74bb16290 0x55b74bacf790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.381+0000 7f32e1d9b640 1 -- 172.21.15.44:0/2131423657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74bad1ec0 msgr2=0x55b74bb36530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.381+0000 7f32e1d9b640 1 --2- 172.21.15.44:0/2131423657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74bad1ec0 0x55b74bb36530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.381+0000 7f32e1d9b640 1 -- 172.21.15.44:0/2131423657 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b74baf6530 con 0x55b74bb10e40 2024-04-22T06:32:12.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.381+0000 7f32e259c640 1 --2- 172.21.15.44:0/2131423657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74bad1ec0 0x55b74bb36530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:32:12.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.381+0000 7f32e1d9b640 1 --2- 172.21.15.44:0/2131423657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74bb10e40 0x55b74bacf250 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f32e4151300 tx=0x7f32d4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5da972fecc49d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:12.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.382+0000 7f32c37fe640 1 -- 172.21.15.44:0/2131423657 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d402c070 con 0x55b74bb10e40 2024-04-22T06:32:12.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.382+0000 7f32c37fe640 1 -- 172.21.15.44:0/2131423657 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32d4003610 con 0x55b74bb10e40 2024-04-22T06:32:12.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.382+0000 7f32c37fe640 1 -- 172.21.15.44:0/2131423657 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d400f040 con 0x55b74bb10e40 2024-04-22T06:32:12.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.382+0000 7f32eb99e740 1 -- 172.21.15.44:0/2131423657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74bb10e40 msgr2=0x55b74bacf250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.382+0000 7f32eb99e740 1 --2- 172.21.15.44:0/2131423657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74bb10e40 0x55b74bacf250 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f32e4151300 tx=0x7f32d4014000 comp rx=0 tx=0).stop 2024-04-22T06:32:12.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.382+0000 7f32eb99e740 1 -- 172.21.15.44:0/2131423657 shutdown_connections 2024-04-22T06:32:12.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.382+0000 7f32eb99e740 1 --2- 172.21.15.44:0/2131423657 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b74bb16290 0x55b74bacf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.382+0000 7f32eb99e740 1 --2- 172.21.15.44:0/2131423657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74bb10e40 0x55b74bacf250 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.382+0000 7f32eb99e740 1 --2- 172.21.15.44:0/2131423657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74bad1ec0 0x55b74bb36530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.382+0000 7f32eb99e740 1 -- 172.21.15.44:0/2131423657 >> 172.21.15.44:0/2131423657 conn(0x55b74bacdca0 msgr2=0x55b74baeead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:12.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.382+0000 7f32eb99e740 1 -- 172.21.15.44:0/2131423657 shutdown_connections 2024-04-22T06:32:12.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.382+0000 7f32eb99e740 1 -- 172.21.15.44:0/2131423657 wait complete. 2024-04-22T06:32:12.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.383+0000 7f32eb99e740 1 -- start start 2024-04-22T06:32:12.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.383+0000 7f32eb99e740 1 Processor -- start 2024-04-22T06:32:12.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.384+0000 7f32eb99e740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b74bad1ec0 0x55b74bb14360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.384+0000 7f32eb99e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74bb10e40 0x55b74baf04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.384+0000 7f32eb99e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74bb16290 0x55b74bb19c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.384+0000 7f32eb99e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b74baef500 con 0x55b74bb10e40 2024-04-22T06:32:12.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.384+0000 7f32eb99e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55b74bae5090 con 0x55b74bb16290 2024-04-22T06:32:12.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.384+0000 7f32eb99e740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55b74bb0a890 con 0x55b74bad1ec0 2024-04-22T06:32:12.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.384+0000 7f32eb99e740 10 client.0 _ll_register_callbacks cb 0x7f32d8059c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T06:32:12.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.384+0000 7f32e1d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74bb10e40 0x55b74baf04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.384+0000 7f32e1d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74bb10e40 0x55b74baf04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36626/0 (socket says 172.21.15.44:36626) 2024-04-22T06:32:12.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.384+0000 7f32e1d9b640 1 -- 172.21.15.44:0/1128605235 learned_addr learned my addr 172.21.15.44:0/1128605235 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:12.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.384+0000 7f32e259c640 1 --2- 172.21.15.44:0/1128605235 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b74bad1ec0 0x55b74bb14360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.384+0000 7f32e2d9d640 1 --2- 172.21.15.44:0/1128605235 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74bb16290 0x55b74bb19c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.384+0000 7f32e1d9b640 1 -- 172.21.15.44:0/1128605235 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b74bad1ec0 msgr2=0x55b74bb14360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.384+0000 7f32e1d9b640 1 --2- 172.21.15.44:0/1128605235 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b74bad1ec0 0x55b74bb14360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.384+0000 7f32e1d9b640 1 -- 172.21.15.44:0/1128605235 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74bb16290 msgr2=0x55b74bb19c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.384+0000 7f32e1d9b640 1 --2- 172.21.15.44:0/1128605235 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74bb16290 0x55b74bb19c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.384+0000 7f32e1d9b640 1 -- 172.21.15.44:0/1128605235 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b74bb0fc70 con 0x55b74bb10e40 2024-04-22T06:32:12.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.384+0000 7f32e1d9b640 1 --2- 172.21.15.44:0/1128605235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74bb10e40 0x55b74baf04b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f32d4017000 tx=0x7f32d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3508f874ef2450 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:12.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.384+0000 7f32c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T06:32:12.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.384+0000 7f32c17fa640 1 -- 172.21.15.44:0/1128605235 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d4035060 con 0x55b74bb10e40 2024-04-22T06:32:12.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.384+0000 7f32c17fa640 1 -- 172.21.15.44:0/1128605235 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f32d4003a80 con 0x55b74bb10e40 2024-04-22T06:32:12.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.384+0000 7f32eb99e740 1 -- 172.21.15.44:0/1128605235 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32d4003610 con 0x55b74bb10e40 2024-04-22T06:32:12.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.385+0000 7f32c17fa640 1 -- 172.21.15.44:0/1128605235 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f32d403b050 con 0x55b74bb10e40 2024-04-22T06:32:12.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.385+0000 7f32eb99e740 10 client.5214 fetch_fsmap learned FSMap version 18 2024-04-22T06:32:12.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.385+0000 7f32eb99e740 1 -- 172.21.15.44:0/1128605235 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55b74baf5bb0 con 0x55b74bb10e40 2024-04-22T06:32:12.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.385+0000 7f32c17fa640 1 -- 172.21.15.44:0/1128605235 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32d4003610 con 0x55b74bb10e40 2024-04-22T06:32:12.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.385+0000 7f32c17fa640 1 -- 172.21.15.44:0/1128605235 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 18) ==== 4792+0+0 (secure 0 0 0) 0x7f32d403e020 con 0x55b74bb10e40 2024-04-22T06:32:12.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.385+0000 7f32eb99e740 10 client.5214 fetch_fsmap finished waiting for FSMap version 18 2024-04-22T06:32:12.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.385+0000 7f32eb99e740 10 client.5214 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-22T06:32:12.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.385+0000 7f32eb99e740 20 client.5214 populate_metadata read hostname 'smithi044' 2024-04-22T06:32:12.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.385+0000 7f32eb99e740 1 --2- 172.21.15.44:0/1128605235 >> [v2:172.21.15.44:6833/2383621014,v1:172.21.15.44:6835/2383621014] conn(0x55b74bb89610 0x55b74bb98de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.385+0000 7f32eb99e740 4 client.5214 mds_command: new command op to 4290 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T06:32:12.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.385+0000 7f32eb99e740 1 -- 172.21.15.44:0/1128605235 --> [v2:172.21.15.44:6833/2383621014,v1:172.21.15.44:6835/2383621014] -- command(tid 0: {"prefix": "status"}) -- 0x55b74ba3c3a0 con 0x55b74bb89610 2024-04-22T06:32:12.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.385+0000 7f32e259c640 1 --2- 172.21.15.44:0/1128605235 >> [v2:172.21.15.44:6833/2383621014,v1:172.21.15.44:6835/2383621014] conn(0x55b74bb89610 0x55b74bb98de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.385+0000 7f32c17fa640 10 client.5214 ms_handle_connect on v2:172.21.15.44:6833/2383621014 2024-04-22T06:32:12.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.385+0000 7f32e259c640 1 --2- 172.21.15.44:0/1128605235 >> [v2:172.21.15.44:6833/2383621014,v1:172.21.15.44:6835/2383621014] conn(0x55b74bb89610 0x55b74bb98de0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=840d3bb771647c8a server_cookie=ea87ee8f5a6226b0 in_seq=0 out_seq=0 2024-04-22T06:32:12.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.386+0000 7f32c17fa640 1 -- 172.21.15.44:0/1128605235 <== mds.3 v2:172.21.15.44:6833/2383621014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55b74ba3c3a0 con 0x55b74bb89610 2024-04-22T06:32:12.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.386+0000 7f32c17fa640 10 client.5214 handle_command_reply: tid=0 2024-04-22T06:32:12.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.386+0000 7f32eb99e740 1 client.5214 shutdown 2024-04-22T06:32:12.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.386+0000 7f32eb99e740 1 -- 172.21.15.44:0/1128605235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74bb10e40 msgr2=0x55b74baf04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.386+0000 7f32eb99e740 1 --2- 172.21.15.44:0/1128605235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74bb10e40 0x55b74baf04b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f32d4017000 tx=0x7f32d4004730 comp rx=0 tx=0).stop 2024-04-22T06:32:12.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.386+0000 7f32eb99e740 1 -- 172.21.15.44:0/1128605235 shutdown_connections 2024-04-22T06:32:12.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.386+0000 7f32eb99e740 1 --2- 172.21.15.44:0/1128605235 >> [v2:172.21.15.44:6833/2383621014,v1:172.21.15.44:6835/2383621014] conn(0x55b74bb89610 0x55b74bb98de0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.386+0000 7f32eb99e740 1 --2- 172.21.15.44:0/1128605235 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b74bb16290 0x55b74bb19c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.386+0000 7f32eb99e740 1 --2- 172.21.15.44:0/1128605235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b74bb10e40 0x55b74baf04b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.386+0000 7f32c17fa640 0 client.5214 ms_handle_reset on v2:172.21.15.44:6833/2383621014 2024-04-22T06:32:12.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.386+0000 7f32eb99e740 1 --2- 172.21.15.44:0/1128605235 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b74bad1ec0 0x55b74bb14360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.386+0000 7f32eb99e740 1 -- 172.21.15.44:0/1128605235 >> 172.21.15.44:0/1128605235 conn(0x55b74bacdca0 msgr2=0x55b74baee8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:12.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.386+0000 7f32eb99e740 1 -- 172.21.15.44:0/1128605235 shutdown_connections 2024-04-22T06:32:12.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.386+0000 7f32eb99e740 1 -- 172.21.15.44:0/1128605235 wait complete. 2024-04-22T06:32:12.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.386+0000 7f32eb99e740 20 client.5214 trim_cache size 0 max 16384 2024-04-22T06:32:12.392 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-22T06:32:12.392 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "9b0a34ca-403d-4513-8a66-d3749ba559aa", 2024-04-22T06:32:12.392 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 3, 2024-04-22T06:32:12.392 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 4290, 2024-04-22T06:32:12.392 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-04-22T06:32:12.392 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-04-22T06:32:12.392 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-04-22T06:32:12.392 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 28.254303091000001, 2024-04-22T06:32:12.392 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 18, 2024-04-22T06:32:12.392 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 27, 2024-04-22T06:32:12.392 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 27, 2024-04-22T06:32:12.393 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 72.294266366000002 2024-04-22T06:32:12.393 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-22T06:32:12.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.386+0000 7f32e8cd2640 1 -- 172.21.15.44:0/1753929760 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f32c406c8f0 msgr2=0x7f32c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.386+0000 7f32e8cd2640 1 --2- 172.21.15.44:0/1753929760 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f32c406c8f0 0x7f32c406edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f32d4006fd0 tx=0x7f32d4008040 comp rx=0 tx=0).stop 2024-04-22T06:32:12.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.386+0000 7f32e8cd2640 1 -- 172.21.15.44:0/1753929760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32e406d470 msgr2=0x7f32e40cfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.386+0000 7f32e8cd2640 1 --2- 172.21.15.44:0/1753929760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32e406d470 0x7f32e40cfdb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f32d8007e60 tx=0x7f32d800e4b0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.387+0000 7f32e8cd2640 1 -- 172.21.15.44:0/1753929760 shutdown_connections 2024-04-22T06:32:12.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.387+0000 7f32e8cd2640 1 --2- 172.21.15.44:0/1753929760 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f32c406c8f0 0x7f32c406edb0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.387+0000 7f32e8cd2640 1 --2- 172.21.15.44:0/1753929760 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32e41552e0 0x7f32e40d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.387+0000 7f32e8cd2640 1 --2- 172.21.15.44:0/1753929760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32e4154f00 0x7f32e41f3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.387+0000 7f32e8cd2640 1 --2- 172.21.15.44:0/1753929760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32e406d470 0x7f32e40cfdb0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.387+0000 7f32e8cd2640 1 -- 172.21.15.44:0/1753929760 >> 172.21.15.44:0/1753929760 conn(0x7f32e40733c0 msgr2=0x7f32e4151450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:12.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.387+0000 7f32e8cd2640 1 -- 172.21.15.44:0/1753929760 shutdown_connections 2024-04-22T06:32:12.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.387+0000 7f32e8cd2640 1 -- 172.21.15.44:0/1753929760 wait complete. 2024-04-22T06:32:12.396 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-22T06:32:12.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.499+0000 7fe5b808c640 1 Processor -- start 2024-04-22T06:32:12.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.499+0000 7fe5b808c640 1 -- start start 2024-04-22T06:32:12.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.500+0000 7fe5b808c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe5b00cc4a0 0x7fe5b00cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.500+0000 7fe5b808c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe5b006cd30 0x7fe5b006d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.500+0000 7fe5b808c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5b00cc0a0 con 0x7fe5b0155570 2024-04-22T06:32:12.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.500+0000 7fe5b808c640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fe5b006c8a0 con 0x7fe5b00cc4a0 2024-04-22T06:32:12.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.500+0000 7fe5b808c640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe5b015c1d0 con 0x7fe5b006cd30 2024-04-22T06:32:12.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.500+0000 7fe5b5627640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe5b00cc4a0 0x7fe5b00cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.500+0000 7fe5b5627640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe5b00cc4a0 0x7fe5b00cc8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36632/0 (socket says 172.21.15.44:36632) 2024-04-22T06:32:12.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.500+0000 7fe5b5627640 1 -- 172.21.15.44:0/626231581 learned_addr learned my addr 172.21.15.44:0/626231581 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:12.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.500+0000 7fe5b6629640 1 --2- 172.21.15.44:0/626231581 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe5b006cd30 0x7fe5b006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.500+0000 7fe5b5627640 1 -- 172.21.15.44:0/626231581 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe5b006cd30 msgr2=0x7fe5b006d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.500+0000 7fe5b5627640 1 --2- 172.21.15.44:0/626231581 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe5b006cd30 0x7fe5b006d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.500+0000 7fe5b5627640 1 -- 172.21.15.44:0/626231581 >> v1:172.21.15.70:6789/0 conn(0x7fe5b0155570 legacy=0x7fe5b00cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.500+0000 7fe5b5627640 1 -- 172.21.15.44:0/626231581 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5b015b650 con 0x7fe5b00cc4a0 2024-04-22T06:32:12.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.501+0000 7fe5b5627640 1 --2- 172.21.15.44:0/626231581 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5b00cc4a0 0x7fe5b00cc8e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe5b006ca20 tx=0x7fe5a402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39a9995cf481aa54 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:12.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.501+0000 7fe5b4e26640 1 -- 172.21.15.44:0/626231581 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a40317c0 con 0x7fe5b00cc4a0 2024-04-22T06:32:12.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.501+0000 7fe5b4e26640 1 -- 172.21.15.44:0/626231581 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5a4037070 con 0x7fe5b00cc4a0 2024-04-22T06:32:12.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.501+0000 7fe5b4e26640 1 -- 172.21.15.44:0/626231581 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4031b80 con 0x7fe5b00cc4a0 2024-04-22T06:32:12.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.502+0000 7fe5b808c640 1 -- 172.21.15.44:0/626231581 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5b00cc4a0 msgr2=0x7fe5b00cc8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.502+0000 7fe5b808c640 1 --2- 172.21.15.44:0/626231581 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5b00cc4a0 0x7fe5b00cc8e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe5b006ca20 tx=0x7fe5a402b860 comp rx=0 tx=0).stop 2024-04-22T06:32:12.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.502+0000 7fe5b808c640 1 -- 172.21.15.44:0/626231581 shutdown_connections 2024-04-22T06:32:12.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.502+0000 7fe5b808c640 1 --2- 172.21.15.44:0/626231581 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe5b006cd30 0x7fe5b006d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.502+0000 7fe5b808c640 1 --2- 172.21.15.44:0/626231581 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5b00cc4a0 0x7fe5b00cc8e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.502+0000 7fe5b808c640 1 -- 172.21.15.44:0/626231581 >> 172.21.15.44:0/626231581 conn(0x7fe5b0075870 msgr2=0x7fe5b0073020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:12.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.502+0000 7fe5b808c640 1 -- 172.21.15.44:0/626231581 shutdown_connections 2024-04-22T06:32:12.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.502+0000 7fe5b808c640 1 -- 172.21.15.44:0/626231581 wait complete. 2024-04-22T06:32:12.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.503+0000 7fe5b808c640 1 Processor -- start 2024-04-22T06:32:12.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.503+0000 7fe5b808c640 1 -- start start 2024-04-22T06:32:12.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.504+0000 7fe5b808c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5b006cd30 0x7fe5b0158790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.504+0000 7fe5b808c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5b00cc4a0 0x7fe5b0157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.504+0000 7fe5b808c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe5b0155570 0x7fe5b01f48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.504+0000 7fe5b808c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b015bef0 con 0x7fe5b00cc4a0 2024-04-22T06:32:12.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.504+0000 7fe5b808c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b015bc10 con 0x7fe5b006cd30 2024-04-22T06:32:12.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.504+0000 7fe5b808c640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe5b0155950 con 0x7fe5b0155570 2024-04-22T06:32:12.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.504+0000 7fe5b5627640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5b00cc4a0 0x7fe5b0157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.504+0000 7fe5b5627640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5b00cc4a0 0x7fe5b0157ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36648/0 (socket says 172.21.15.44:36648) 2024-04-22T06:32:12.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.504+0000 7fe5b5627640 1 -- 172.21.15.44:0/40800922 learned_addr learned my addr 172.21.15.44:0/40800922 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:12.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.504+0000 7fe5b5e28640 1 --2- 172.21.15.44:0/40800922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5b006cd30 0x7fe5b0158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.504+0000 7fe5b6629640 1 --2- 172.21.15.44:0/40800922 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe5b0155570 0x7fe5b01f48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.504+0000 7fe5b5627640 1 -- 172.21.15.44:0/40800922 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe5b0155570 msgr2=0x7fe5b01f48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.504+0000 7fe5b5627640 1 --2- 172.21.15.44:0/40800922 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe5b0155570 0x7fe5b01f48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.504+0000 7fe5b5627640 1 -- 172.21.15.44:0/40800922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5b006cd30 msgr2=0x7fe5b0158790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.504+0000 7fe5b5627640 1 --2- 172.21.15.44:0/40800922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5b006cd30 0x7fe5b0158790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.504+0000 7fe5b5627640 1 -- 172.21.15.44:0/40800922 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5b01f1cb0 con 0x7fe5b00cc4a0 2024-04-22T06:32:12.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.504+0000 7fe5b5e28640 1 --2- 172.21.15.44:0/40800922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5b006cd30 0x7fe5b0158790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:32:12.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.504+0000 7fe5b5627640 1 --2- 172.21.15.44:0/40800922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5b00cc4a0 0x7fe5b0157ab0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe5a402b8a0 tx=0x7fe5a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:12.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.504+0000 7fe5ae7fc640 1 -- 172.21.15.44:0/40800922 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4032860 con 0x7fe5b00cc4a0 2024-04-22T06:32:12.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.505+0000 7fe5ae7fc640 1 -- 172.21.15.44:0/40800922 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5a4032a00 con 0x7fe5b00cc4a0 2024-04-22T06:32:12.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.505+0000 7fe5b808c640 1 -- 172.21.15.44:0/40800922 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5b01f1190 con 0x7fe5b00cc4a0 2024-04-22T06:32:12.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.505+0000 7fe5ae7fc640 1 -- 172.21.15.44:0/40800922 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4032dd0 con 0x7fe5b00cc4a0 2024-04-22T06:32:12.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.505+0000 7fe5b808c640 1 -- 172.21.15.44:0/40800922 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5b01f19d0 con 0x7fe5b00cc4a0 2024-04-22T06:32:12.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.506+0000 7fe5ae7fc640 1 -- 172.21.15.44:0/40800922 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe5a40453f0 con 0x7fe5b00cc4a0 2024-04-22T06:32:12.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.506+0000 7fe5ae7fc640 1 --2- 172.21.15.44:0/40800922 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fe59c06c8f0 0x7fe59c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.506+0000 7fe5ae7fc640 1 -- 172.21.15.44:0/40800922 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fe5a40ae270 con 0x7fe5b00cc4a0 2024-04-22T06:32:12.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.507+0000 7fe5b5e28640 1 --2- 172.21.15.44:0/40800922 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fe59c06c8f0 0x7fe59c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.507+0000 7fe5b5e28640 1 --2- 172.21.15.44:0/40800922 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fe59c06c8f0 0x7fe59c06edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe5a8006fd0 tx=0x7fe5a8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:12.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.507+0000 7fe5bac81740 1 Processor -- start 2024-04-22T06:32:12.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.508+0000 7fe5bac81740 1 -- start start 2024-04-22T06:32:12.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.508+0000 7fe5bac81740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb37860e40 0x55fb37869f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.508+0000 7fe5bac81740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb37821ec0 0x55fb3786a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.508+0000 7fe5bac81740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fb3781dca0 0x55fb37845ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.508+0000 7fe5bac81740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55fb37784b10 con 0x55fb37860e40 2024-04-22T06:32:12.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.508+0000 7fe5bac81740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55fb37722240 con 0x55fb37821ec0 2024-04-22T06:32:12.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.508+0000 7fe5bac81740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55fb37780ad0 con 0x55fb3781dca0 2024-04-22T06:32:12.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.508+0000 7fe5b5e28640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb37860e40 0x55fb37869f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.508+0000 7fe5b5e28640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb37860e40 0x55fb37869f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36660/0 (socket says 172.21.15.44:36660) 2024-04-22T06:32:12.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.508+0000 7fe5b5e28640 1 -- 172.21.15.44:0/3266335546 learned_addr learned my addr 172.21.15.44:0/3266335546 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:12.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.508+0000 7fe5b6629640 1 --2- 172.21.15.44:0/3266335546 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fb3781dca0 0x55fb37845ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.508+0000 7fe5b5627640 1 --2- 172.21.15.44:0/3266335546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb37821ec0 0x55fb3786a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.509+0000 7fe5b5e28640 1 -- 172.21.15.44:0/3266335546 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fb3781dca0 msgr2=0x55fb37845ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.509+0000 7fe5b5e28640 1 --2- 172.21.15.44:0/3266335546 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fb3781dca0 0x55fb37845ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.509+0000 7fe5b5e28640 1 -- 172.21.15.44:0/3266335546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb37821ec0 msgr2=0x55fb3786a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.509+0000 7fe5b5e28640 1 --2- 172.21.15.44:0/3266335546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb37821ec0 0x55fb3786a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.509+0000 7fe5b5e28640 1 -- 172.21.15.44:0/3266335546 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb374972d0 con 0x55fb37860e40 2024-04-22T06:32:12.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.509+0000 7fe5b5627640 1 --2- 172.21.15.44:0/3266335546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb37821ec0 0x55fb3786a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:32:12.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.509+0000 7fe5b5e28640 1 --2- 172.21.15.44:0/3266335546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb37860e40 0x55fb37869f60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe5a800d8e0 tx=0x7fe5a8010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ed61a3ed98f6e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:12.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.509+0000 7fe5977fe640 1 -- 172.21.15.44:0/3266335546 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a8024070 con 0x55fb37860e40 2024-04-22T06:32:12.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.509+0000 7fe5977fe640 1 -- 172.21.15.44:0/3266335546 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5a80149b0 con 0x55fb37860e40 2024-04-22T06:32:12.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.509+0000 7fe5977fe640 1 -- 172.21.15.44:0/3266335546 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a8014d80 con 0x55fb37860e40 2024-04-22T06:32:12.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.510+0000 7fe5bac81740 1 -- 172.21.15.44:0/3266335546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb37860e40 msgr2=0x55fb37869f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.510+0000 7fe5bac81740 1 --2- 172.21.15.44:0/3266335546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb37860e40 0x55fb37869f60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe5a800d8e0 tx=0x7fe5a8010000 comp rx=0 tx=0).stop 2024-04-22T06:32:12.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.510+0000 7fe5bac81740 1 -- 172.21.15.44:0/3266335546 shutdown_connections 2024-04-22T06:32:12.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.510+0000 7fe5bac81740 1 --2- 172.21.15.44:0/3266335546 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fb3781dca0 0x55fb37845ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.510+0000 7fe5bac81740 1 --2- 172.21.15.44:0/3266335546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb37821ec0 0x55fb3786a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.510+0000 7fe5bac81740 1 --2- 172.21.15.44:0/3266335546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb37860e40 0x55fb37869f60 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.510+0000 7fe5bac81740 1 -- 172.21.15.44:0/3266335546 >> 172.21.15.44:0/3266335546 conn(0x55fb3781c990 msgr2=0x55fb3781f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:12.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.510+0000 7fe5bac81740 1 -- 172.21.15.44:0/3266335546 shutdown_connections 2024-04-22T06:32:12.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.510+0000 7fe5bac81740 1 -- 172.21.15.44:0/3266335546 wait complete. 2024-04-22T06:32:12.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.510+0000 7fe5bac81740 1 -- start start 2024-04-22T06:32:12.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.510+0000 7fe5bac81740 1 Processor -- start 2024-04-22T06:32:12.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.511+0000 7fe5bac81740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb37821ec0 0x55fb37868d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.511+0000 7fe5bac81740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb37860e40 0x55fb3785a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.511+0000 7fe5bac81740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fb37866ff0 0x55fb3785acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.511+0000 7fe5bac81740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55fb37820250 con 0x55fb37821ec0 2024-04-22T06:32:12.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.511+0000 7fe5bac81740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55fb3770d190 con 0x55fb37860e40 2024-04-22T06:32:12.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.511+0000 7fe5bac81740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55fb37820530 con 0x55fb37866ff0 2024-04-22T06:32:12.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.511+0000 7fe5bac81740 10 client.0 _ll_register_callbacks cb 0x55fb377088b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T06:32:12.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.511+0000 7fe5b5e28640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb37821ec0 0x55fb37868d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.511+0000 7fe5b5e28640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb37821ec0 0x55fb37868d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36662/0 (socket says 172.21.15.44:36662) 2024-04-22T06:32:12.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.511+0000 7fe5b5e28640 1 -- 172.21.15.44:0/89016383 learned_addr learned my addr 172.21.15.44:0/89016383 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:12.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.511+0000 7fe5b5627640 1 --2- 172.21.15.44:0/89016383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb37860e40 0x55fb3785a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.511+0000 7fe5b6629640 1 --2- 172.21.15.44:0/89016383 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fb37866ff0 0x55fb3785acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.512+0000 7fe5b5e28640 1 -- 172.21.15.44:0/89016383 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fb37866ff0 msgr2=0x55fb3785acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.512+0000 7fe5b5e28640 1 --2- 172.21.15.44:0/89016383 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fb37866ff0 0x55fb3785acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.512+0000 7fe5b5e28640 1 -- 172.21.15.44:0/89016383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb37860e40 msgr2=0x55fb3785a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.512+0000 7fe5b5e28640 1 --2- 172.21.15.44:0/89016383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb37860e40 0x55fb3785a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.512+0000 7fe5b5e28640 1 -- 172.21.15.44:0/89016383 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb375616b0 con 0x55fb37821ec0 2024-04-22T06:32:12.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.512+0000 7fe5b5e28640 1 --2- 172.21.15.44:0/89016383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb37821ec0 0x55fb37868d60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe5a8017040 tx=0x7fe5a8004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a924566e7ce1e75 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:12.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.512+0000 7fe594ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T06:32:12.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.512+0000 7fe594ff9640 1 -- 172.21.15.44:0/89016383 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a8024040 con 0x55fb37821ec0 2024-04-22T06:32:12.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.512+0000 7fe594ff9640 1 -- 172.21.15.44:0/89016383 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fe5a802a650 con 0x55fb37821ec0 2024-04-22T06:32:12.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.512+0000 7fe5bac81740 1 -- 172.21.15.44:0/89016383 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5a80149b0 con 0x55fb37821ec0 2024-04-22T06:32:12.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.512+0000 7fe594ff9640 1 -- 172.21.15.44:0/89016383 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fe5a8005910 con 0x55fb37821ec0 2024-04-22T06:32:12.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.512+0000 7fe5bac81740 10 client.5226 fetch_fsmap learned FSMap version 18 2024-04-22T06:32:12.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.512+0000 7fe5bac81740 1 -- 172.21.15.44:0/89016383 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55fb37819310 con 0x55fb37821ec0 2024-04-22T06:32:12.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.512+0000 7fe594ff9640 1 -- 172.21.15.44:0/89016383 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5a80149b0 con 0x55fb37821ec0 2024-04-22T06:32:12.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.512+0000 7fe594ff9640 1 -- 172.21.15.44:0/89016383 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 18) ==== 4792+0+0 (secure 0 0 0) 0x7fe5a8013070 con 0x55fb37821ec0 2024-04-22T06:32:12.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.513+0000 7fe5bac81740 10 client.5226 fetch_fsmap finished waiting for FSMap version 18 2024-04-22T06:32:12.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.513+0000 7fe5bac81740 10 client.5226 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-22T06:32:12.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.513+0000 7fe5bac81740 20 client.5226 populate_metadata read hostname 'smithi044' 2024-04-22T06:32:12.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.513+0000 7fe5bac81740 1 --2- 172.21.15.44:0/89016383 >> [v2:172.21.15.44:6836/915960587,v1:172.21.15.44:6837/915960587] conn(0x55fb3781b510 0x55fb378805e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.513+0000 7fe5bac81740 4 client.5226 mds_command: new command op to 4293 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T06:32:12.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.513+0000 7fe5bac81740 1 -- 172.21.15.44:0/89016383 --> [v2:172.21.15.44:6836/915960587,v1:172.21.15.44:6837/915960587] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb37792530 con 0x55fb3781b510 2024-04-22T06:32:12.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.513+0000 7fe5b5627640 1 --2- 172.21.15.44:0/89016383 >> [v2:172.21.15.44:6836/915960587,v1:172.21.15.44:6837/915960587] conn(0x55fb3781b510 0x55fb378805e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.513+0000 7fe5b5627640 1 --2- 172.21.15.44:0/89016383 >> [v2:172.21.15.44:6836/915960587,v1:172.21.15.44:6837/915960587] conn(0x55fb3781b510 0x55fb378805e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2b9325ece4f1ed81 server_cookie=18c85e9c3255c9d0 in_seq=0 out_seq=0 2024-04-22T06:32:12.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.513+0000 7fe594ff9640 10 client.5226 ms_handle_connect on v2:172.21.15.44:6836/915960587 2024-04-22T06:32:12.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.514+0000 7fe594ff9640 1 -- 172.21.15.44:0/89016383 <== mds.2 v2:172.21.15.44:6836/915960587 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fb37792530 con 0x55fb3781b510 2024-04-22T06:32:12.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.514+0000 7fe594ff9640 10 client.5226 handle_command_reply: tid=0 2024-04-22T06:32:12.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.514+0000 7fe5bac81740 1 client.5226 shutdown 2024-04-22T06:32:12.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.514+0000 7fe5bac81740 1 -- 172.21.15.44:0/89016383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb37821ec0 msgr2=0x55fb37868d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.514+0000 7fe5bac81740 1 --2- 172.21.15.44:0/89016383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb37821ec0 0x55fb37868d60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe5a8017040 tx=0x7fe5a8004680 comp rx=0 tx=0).stop 2024-04-22T06:32:12.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.514+0000 7fe5bac81740 1 -- 172.21.15.44:0/89016383 shutdown_connections 2024-04-22T06:32:12.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.514+0000 7fe5bac81740 1 --2- 172.21.15.44:0/89016383 >> [v2:172.21.15.44:6836/915960587,v1:172.21.15.44:6837/915960587] conn(0x55fb3781b510 0x55fb378805e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.514+0000 7fe5bac81740 1 --2- 172.21.15.44:0/89016383 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fb37866ff0 0x55fb3785acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.514+0000 7fe5bac81740 1 --2- 172.21.15.44:0/89016383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb37860e40 0x55fb3785a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.514+0000 7fe594ff9640 0 client.5226 ms_handle_reset on v2:172.21.15.44:6836/915960587 2024-04-22T06:32:12.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.514+0000 7fe5bac81740 1 --2- 172.21.15.44:0/89016383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb37821ec0 0x55fb37868d60 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.514+0000 7fe5bac81740 1 -- 172.21.15.44:0/89016383 >> 172.21.15.44:0/89016383 conn(0x55fb3781dca0 msgr2=0x55fb37845dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:12.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.514+0000 7fe5bac81740 1 -- 172.21.15.44:0/89016383 shutdown_connections 2024-04-22T06:32:12.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.514+0000 7fe5bac81740 1 -- 172.21.15.44:0/89016383 wait complete. 2024-04-22T06:32:12.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.514+0000 7fe5bac81740 20 client.5226 trim_cache size 0 max 16384 2024-04-22T06:32:12.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.532+0000 7fe5bac81740 1 Processor -- start 2024-04-22T06:32:12.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.532+0000 7fe5bac81740 1 -- start start 2024-04-22T06:32:12.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.533+0000 7fe5bac81740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fb37821ec0 0x55fb37844db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.533+0000 7fe5bac81740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb37860e40 0x55fb378452f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.533+0000 7fe5bac81740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb37885840 0x55fb37885cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.533+0000 7fe5bac81740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b0150ff0 con 0x55fb37885840 2024-04-22T06:32:12.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.533+0000 7fe5bac81740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55fb37819650 con 0x55fb37860e40 2024-04-22T06:32:12.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.533+0000 7fe5bac81740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55fb37874f90 con 0x55fb37821ec0 2024-04-22T06:32:12.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.533+0000 7fe5b6629640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb37885840 0x55fb37885cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.533+0000 7fe5b6629640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb37885840 0x55fb37885cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36674/0 (socket says 172.21.15.44:36674) 2024-04-22T06:32:12.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.533+0000 7fe5b6629640 1 -- 172.21.15.44:0/274150375 learned_addr learned my addr 172.21.15.44:0/274150375 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:12.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.533+0000 7fe5b5627640 1 --2- 172.21.15.44:0/274150375 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb37860e40 0x55fb378452f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.533+0000 7fe5b5e28640 1 --2- 172.21.15.44:0/274150375 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fb37821ec0 0x55fb37844db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.533+0000 7fe5b6629640 1 -- 172.21.15.44:0/274150375 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fb37821ec0 msgr2=0x55fb37844db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.533+0000 7fe5b6629640 1 --2- 172.21.15.44:0/274150375 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fb37821ec0 0x55fb37844db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.533+0000 7fe5b6629640 1 -- 172.21.15.44:0/274150375 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb37860e40 msgr2=0x55fb378452f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.533+0000 7fe5b6629640 1 --2- 172.21.15.44:0/274150375 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb37860e40 0x55fb378452f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.533+0000 7fe5b6629640 1 -- 172.21.15.44:0/274150375 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb37867530 con 0x55fb37885840 2024-04-22T06:32:12.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.533+0000 7fe5b5627640 1 --2- 172.21.15.44:0/274150375 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb37860e40 0x55fb378452f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:32:12.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.534+0000 7fe5b6629640 1 --2- 172.21.15.44:0/274150375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb37885840 0x55fb37885cc0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe5a000f4e0 tx=0x7fe5a000f9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aadf9cd1d1e68dd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:12.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.534+0000 7fe596ffd640 1 -- 172.21.15.44:0/274150375 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a000ea40 con 0x55fb37885840 2024-04-22T06:32:12.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.534+0000 7fe596ffd640 1 -- 172.21.15.44:0/274150375 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5a001e070 con 0x55fb37885840 2024-04-22T06:32:12.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.534+0000 7fe596ffd640 1 -- 172.21.15.44:0/274150375 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a000ee00 con 0x55fb37885840 2024-04-22T06:32:12.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.534+0000 7fe5bac81740 1 -- 172.21.15.44:0/274150375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb37885840 msgr2=0x55fb37885cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.534+0000 7fe5bac81740 1 --2- 172.21.15.44:0/274150375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb37885840 0x55fb37885cc0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe5a000f4e0 tx=0x7fe5a000f9a0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.534+0000 7fe5bac81740 1 -- 172.21.15.44:0/274150375 shutdown_connections 2024-04-22T06:32:12.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.534+0000 7fe5bac81740 1 --2- 172.21.15.44:0/274150375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb37885840 0x55fb37885cc0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.534+0000 7fe5bac81740 1 --2- 172.21.15.44:0/274150375 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb37860e40 0x55fb378452f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.534+0000 7fe5bac81740 1 --2- 172.21.15.44:0/274150375 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fb37821ec0 0x55fb37844db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.534+0000 7fe5bac81740 1 -- 172.21.15.44:0/274150375 >> 172.21.15.44:0/274150375 conn(0x55fb3781dca0 msgr2=0x55fb37815800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:12.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.534+0000 7fe5bac81740 1 -- 172.21.15.44:0/274150375 shutdown_connections 2024-04-22T06:32:12.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.534+0000 7fe5bac81740 1 -- 172.21.15.44:0/274150375 wait complete. 2024-04-22T06:32:12.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.535+0000 7fe5bac81740 1 -- start start 2024-04-22T06:32:12.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.535+0000 7fe5bac81740 1 Processor -- start 2024-04-22T06:32:12.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.536+0000 7fe5bac81740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fb37821ec0 0x55fb3781e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.536+0000 7fe5bac81740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb37860e40 0x55fb378d5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.536+0000 7fe5bac81740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb37885840 0x55fb37844db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.536+0000 7fe5bac81740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55fb37869780 con 0x55fb37860e40 2024-04-22T06:32:12.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.536+0000 7fe5bac81740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55fb3783f540 con 0x55fb37885840 2024-04-22T06:32:12.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.536+0000 7fe5bac81740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55fb37865b10 con 0x55fb37821ec0 2024-04-22T06:32:12.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.536+0000 7fe5bac81740 10 client.0 _ll_register_callbacks cb 0x7fe5a8022070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T06:32:12.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.536+0000 7fe5b5627640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb37860e40 0x55fb378d5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.536+0000 7fe5b5627640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb37860e40 0x55fb378d5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36686/0 (socket says 172.21.15.44:36686) 2024-04-22T06:32:12.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.536+0000 7fe5b5627640 1 -- 172.21.15.44:0/1902906172 learned_addr learned my addr 172.21.15.44:0/1902906172 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:12.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.536+0000 7fe5b5e28640 1 --2- 172.21.15.44:0/1902906172 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fb37821ec0 0x55fb3781e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.536+0000 7fe5b6629640 1 --2- 172.21.15.44:0/1902906172 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb37885840 0x55fb37844db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.536+0000 7fe5b5627640 1 -- 172.21.15.44:0/1902906172 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fb37821ec0 msgr2=0x55fb3781e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.536+0000 7fe5b5627640 1 --2- 172.21.15.44:0/1902906172 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fb37821ec0 0x55fb3781e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.536+0000 7fe5b5627640 1 -- 172.21.15.44:0/1902906172 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb37885840 msgr2=0x55fb37844db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.536+0000 7fe5b5627640 1 --2- 172.21.15.44:0/1902906172 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb37885840 0x55fb37844db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.536+0000 7fe5b5627640 1 -- 172.21.15.44:0/1902906172 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb3781f550 con 0x55fb37860e40 2024-04-22T06:32:12.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.536+0000 7fe5b5627640 1 --2- 172.21.15.44:0/1902906172 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb37860e40 0x55fb378d5ef0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4000c00 tx=0x7fe5a4047040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86a98aa72ef7def2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:12.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.536+0000 7fe594ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T06:32:12.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.537+0000 7fe594ff9640 1 -- 172.21.15.44:0/1902906172 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a407fe60 con 0x55fb37860e40 2024-04-22T06:32:12.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.537+0000 7fe594ff9640 1 -- 172.21.15.44:0/1902906172 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fe5a4078120 con 0x55fb37860e40 2024-04-22T06:32:12.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.537+0000 7fe5bac81740 1 -- 172.21.15.44:0/1902906172 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5a001e070 con 0x55fb37860e40 2024-04-22T06:32:12.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.537+0000 7fe594ff9640 1 -- 172.21.15.44:0/1902906172 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fe5a40a0050 con 0x55fb37860e40 2024-04-22T06:32:12.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.537+0000 7fe5bac81740 10 client.5232 fetch_fsmap learned FSMap version 18 2024-04-22T06:32:12.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.537+0000 7fe5bac81740 1 -- 172.21.15.44:0/1902906172 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55fb37866a80 con 0x55fb37860e40 2024-04-22T06:32:12.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.537+0000 7fe594ff9640 1 -- 172.21.15.44:0/1902906172 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5a001e070 con 0x55fb37860e40 2024-04-22T06:32:12.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.537+0000 7fe594ff9640 1 -- 172.21.15.44:0/1902906172 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 18) ==== 4792+0+0 (secure 0 0 0) 0x7fe5a407bd10 con 0x55fb37860e40 2024-04-22T06:32:12.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.537+0000 7fe5bac81740 10 client.5232 fetch_fsmap finished waiting for FSMap version 18 2024-04-22T06:32:12.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.537+0000 7fe5bac81740 10 client.5232 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-22T06:32:12.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.537+0000 7fe5bac81740 20 client.5232 populate_metadata read hostname 'smithi044' 2024-04-22T06:32:12.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.537+0000 7fe5bac81740 1 --2- 172.21.15.44:0/1902906172 >> [v2:172.21.15.44:6836/915960587,v1:172.21.15.44:6837/915960587] conn(0x55fb3785daf0 0x55fb3785ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.537+0000 7fe5bac81740 4 client.5232 mds_command: new command op to 4293 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T06:32:12.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.537+0000 7fe5bac81740 1 -- 172.21.15.44:0/1902906172 --> [v2:172.21.15.44:6836/915960587,v1:172.21.15.44:6837/915960587] -- command(tid 0: {"prefix": "status"}) -- 0x55fb3778c3a0 con 0x55fb3785daf0 2024-04-22T06:32:12.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.537+0000 7fe5b5e28640 1 --2- 172.21.15.44:0/1902906172 >> [v2:172.21.15.44:6836/915960587,v1:172.21.15.44:6837/915960587] conn(0x55fb3785daf0 0x55fb3785ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.538+0000 7fe594ff9640 10 client.5232 ms_handle_connect on v2:172.21.15.44:6836/915960587 2024-04-22T06:32:12.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.538+0000 7fe5b5e28640 1 --2- 172.21.15.44:0/1902906172 >> [v2:172.21.15.44:6836/915960587,v1:172.21.15.44:6837/915960587] conn(0x55fb3785daf0 0x55fb3785ded0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=363957170424ac91 server_cookie=b65e85f415e15cc9 in_seq=0 out_seq=0 2024-04-22T06:32:12.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.538+0000 7fe594ff9640 1 -- 172.21.15.44:0/1902906172 <== mds.2 v2:172.21.15.44:6836/915960587 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55fb3778c3a0 con 0x55fb3785daf0 2024-04-22T06:32:12.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.538+0000 7fe594ff9640 10 client.5232 handle_command_reply: tid=0 2024-04-22T06:32:12.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.538+0000 7fe5bac81740 1 client.5232 shutdown 2024-04-22T06:32:12.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.538+0000 7fe5bac81740 1 -- 172.21.15.44:0/1902906172 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb37860e40 msgr2=0x55fb378d5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.538+0000 7fe5bac81740 1 --2- 172.21.15.44:0/1902906172 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb37860e40 0x55fb378d5ef0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4000c00 tx=0x7fe5a4047040 comp rx=0 tx=0).stop 2024-04-22T06:32:12.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.538+0000 7fe5bac81740 1 -- 172.21.15.44:0/1902906172 shutdown_connections 2024-04-22T06:32:12.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.538+0000 7fe5bac81740 1 --2- 172.21.15.44:0/1902906172 >> [v2:172.21.15.44:6836/915960587,v1:172.21.15.44:6837/915960587] conn(0x55fb3785daf0 0x55fb3785ded0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.538+0000 7fe5bac81740 1 --2- 172.21.15.44:0/1902906172 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb37885840 0x55fb37844db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.538+0000 7fe5bac81740 1 --2- 172.21.15.44:0/1902906172 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fb37860e40 0x55fb378d5ef0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.538+0000 7fe5bac81740 1 --2- 172.21.15.44:0/1902906172 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fb37821ec0 0x55fb3781e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.538+0000 7fe594ff9640 0 client.5232 ms_handle_reset on v2:172.21.15.44:6836/915960587 2024-04-22T06:32:12.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.538+0000 7fe5bac81740 1 -- 172.21.15.44:0/1902906172 >> 172.21.15.44:0/1902906172 conn(0x55fb3781dca0 msgr2=0x55fb37820970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:12.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.538+0000 7fe5bac81740 1 -- 172.21.15.44:0/1902906172 shutdown_connections 2024-04-22T06:32:12.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.538+0000 7fe5bac81740 1 -- 172.21.15.44:0/1902906172 wait complete. 2024-04-22T06:32:12.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.538+0000 7fe5bac81740 20 client.5232 trim_cache size 0 max 16384 2024-04-22T06:32:12.545 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-22T06:32:12.545 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "9b0a34ca-403d-4513-8a66-d3749ba559aa", 2024-04-22T06:32:12.545 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 2, 2024-04-22T06:32:12.545 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 4293, 2024-04-22T06:32:12.545 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-04-22T06:32:12.545 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-04-22T06:32:12.545 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-04-22T06:32:12.545 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 34.429580274000003, 2024-04-22T06:32:12.545 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 18, 2024-04-22T06:32:12.545 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 27, 2024-04-22T06:32:12.546 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 27, 2024-04-22T06:32:12.546 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 72.440281537999994 2024-04-22T06:32:12.546 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-22T06:32:12.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.539+0000 7fe5b808c640 1 -- 172.21.15.44:0/40800922 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fe59c06c8f0 msgr2=0x7fe59c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.539+0000 7fe5b808c640 1 --2- 172.21.15.44:0/40800922 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fe59c06c8f0 0x7fe59c06edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe5a8006fd0 tx=0x7fe5a8008040 comp rx=0 tx=0).stop 2024-04-22T06:32:12.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.539+0000 7fe5b808c640 1 -- 172.21.15.44:0/40800922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5b00cc4a0 msgr2=0x7fe5b0157ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.539+0000 7fe5b808c640 1 --2- 172.21.15.44:0/40800922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5b00cc4a0 0x7fe5b0157ab0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe5a402b8a0 tx=0x7fe5a40046c0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.539+0000 7fe5b808c640 1 -- 172.21.15.44:0/40800922 shutdown_connections 2024-04-22T06:32:12.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.539+0000 7fe5b808c640 1 --2- 172.21.15.44:0/40800922 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fe59c06c8f0 0x7fe59c06edb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.539+0000 7fe5b808c640 1 --2- 172.21.15.44:0/40800922 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe5b0155570 0x7fe5b01f48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.539+0000 7fe5b808c640 1 --2- 172.21.15.44:0/40800922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5b00cc4a0 0x7fe5b0157ab0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.539+0000 7fe5b808c640 1 --2- 172.21.15.44:0/40800922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5b006cd30 0x7fe5b0158790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.539+0000 7fe5b808c640 1 -- 172.21.15.44:0/40800922 >> 172.21.15.44:0/40800922 conn(0x7fe5b0075870 msgr2=0x7fe5b0152b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:12.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.539+0000 7fe5b808c640 1 -- 172.21.15.44:0/40800922 shutdown_connections 2024-04-22T06:32:12.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.539+0000 7fe5b808c640 1 -- 172.21.15.44:0/40800922 wait complete. 2024-04-22T06:32:12.549 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-22T06:32:12.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.652+0000 7f12e4db2640 1 Processor -- start 2024-04-22T06:32:12.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.652+0000 7f12e4db2640 1 -- start start 2024-04-22T06:32:12.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.652+0000 7f12e4db2640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f12e00cc4a0 0x7f12e00cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.653+0000 7f12e4db2640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12e00cc0a0 con 0x7f12e0155570 2024-04-22T06:32:12.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.653+0000 7f12e4db2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12e015b650 con 0x7f12e006cd30 2024-04-22T06:32:12.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.653+0000 7f12e4db2640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f12e006c8a0 con 0x7f12e00cc4a0 2024-04-22T06:32:12.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.653+0000 7f12d7fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f12e00cc4a0 0x7f12e00cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.653+0000 7f12de59c640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f12e0155570 0x7f12e00cb990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:55540/0 (socket says 172.21.15.44:55540) 2024-04-22T06:32:12.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.653+0000 7f12de59c640 1 -- 172.21.15.44:0/3654223779 learned_addr learned my addr 172.21.15.44:0/3654223779 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:32:12.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.653+0000 7f12d7fff640 1 -- 172.21.15.44:0/3654223779 >> v1:172.21.15.70:6789/0 conn(0x7f12e006cd30 legacy=0x7f12e0165ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.653+0000 7f12d7fff640 1 -- 172.21.15.44:0/3654223779 >> v1:172.21.15.44:6789/0 conn(0x7f12e0155570 legacy=0x7f12e00cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.653+0000 7f12ddd9b640 1 -- 172.21.15.44:0/3654223779 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3460553184 0 0) 0x7f12d0001110 con 0x7f12e0155570 2024-04-22T06:32:12.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.653+0000 7f12d7fff640 1 -- 172.21.15.44:0/3654223779 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e015b930 con 0x7f12e00cc4a0 2024-04-22T06:32:12.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.654+0000 7f12d7fff640 1 --2- 172.21.15.44:0/3654223779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12e00cc4a0 0x7f12e00cc8e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f12c80097a0 tx=0x7f12c802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d649f45049693d11 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:12.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.654+0000 7f12ddd9b640 1 -- 172.21.15.44:0/3654223779 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c8035020 con 0x7f12e00cc4a0 2024-04-22T06:32:12.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.654+0000 7f12ddd9b640 1 -- 172.21.15.44:0/3654223779 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12c8031830 con 0x7f12e00cc4a0 2024-04-22T06:32:12.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.654+0000 7f12ddd9b640 1 -- 172.21.15.44:0/3654223779 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c8031c00 con 0x7f12e00cc4a0 2024-04-22T06:32:12.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.654+0000 7f12e4db2640 1 -- 172.21.15.44:0/3654223779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12e00cc4a0 msgr2=0x7f12e00cc8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.654+0000 7f12e4db2640 1 --2- 172.21.15.44:0/3654223779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12e00cc4a0 0x7f12e00cc8e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f12c80097a0 tx=0x7f12c802b8d0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.654+0000 7f12e4db2640 1 -- 172.21.15.44:0/3654223779 shutdown_connections 2024-04-22T06:32:12.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.654+0000 7f12e4db2640 1 --2- 172.21.15.44:0/3654223779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12e00cc4a0 0x7f12e00cc8e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.654+0000 7f12e4db2640 1 -- 172.21.15.44:0/3654223779 >> 172.21.15.44:0/3654223779 conn(0x7f12e0075870 msgr2=0x7f12e0073020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:12.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.655+0000 7f12e4db2640 1 -- 172.21.15.44:0/3654223779 shutdown_connections 2024-04-22T06:32:12.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.655+0000 7f12e4db2640 1 -- 172.21.15.44:0/3654223779 wait complete. 2024-04-22T06:32:12.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.656+0000 7f12e4db2640 1 Processor -- start 2024-04-22T06:32:12.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.656+0000 7f12e4db2640 1 -- start start 2024-04-22T06:32:12.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.656+0000 7f12e4db2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12e006cd30 0x7f12e0158290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.656+0000 7f12e4db2640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f12e00cc4a0 0x7f12e01e58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.656+0000 7f12e4db2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12e0155570 0x7f12e01e6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.656+0000 7f12e4db2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f12e015bef0 con 0x7f12e0155570 2024-04-22T06:32:12.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.656+0000 7f12e4db2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f12e015c1d0 con 0x7f12e006cd30 2024-04-22T06:32:12.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.656+0000 7f12e4db2640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f12e0155950 con 0x7f12e00cc4a0 2024-04-22T06:32:12.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.657+0000 7f12ded9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12e0155570 0x7f12e01e6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.657+0000 7f12ded9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12e0155570 0x7f12e01e6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36696/0 (socket says 172.21.15.44:36696) 2024-04-22T06:32:12.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.657+0000 7f12ded9d640 1 -- 172.21.15.44:0/3419627954 learned_addr learned my addr 172.21.15.44:0/3419627954 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:12.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.657+0000 7f12de59c640 1 --2- 172.21.15.44:0/3419627954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12e006cd30 0x7f12e0158290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.657+0000 7f12d7fff640 1 --2- 172.21.15.44:0/3419627954 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f12e00cc4a0 0x7f12e01e58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.657+0000 7f12ded9d640 1 -- 172.21.15.44:0/3419627954 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f12e00cc4a0 msgr2=0x7f12e01e58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.657+0000 7f12ded9d640 1 --2- 172.21.15.44:0/3419627954 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f12e00cc4a0 0x7f12e01e58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.657+0000 7f12ded9d640 1 -- 172.21.15.44:0/3419627954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12e006cd30 msgr2=0x7f12e0158290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.657+0000 7f12ded9d640 1 --2- 172.21.15.44:0/3419627954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12e006cd30 0x7f12e0158290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.657+0000 7f12ded9d640 1 -- 172.21.15.44:0/3419627954 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e0157ad0 con 0x7f12e0155570 2024-04-22T06:32:12.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.657+0000 7f12ded9d640 1 --2- 172.21.15.44:0/3419627954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12e0155570 0x7f12e01e6110 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f12d800c7a0 tx=0x7f12d800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:12.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.657+0000 7f12d6ffd640 1 -- 172.21.15.44:0/3419627954 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d8003b70 con 0x7f12e0155570 2024-04-22T06:32:12.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.657+0000 7f12d6ffd640 1 -- 172.21.15.44:0/3419627954 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12d8003d30 con 0x7f12e0155570 2024-04-22T06:32:12.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.657+0000 7f12d6ffd640 1 -- 172.21.15.44:0/3419627954 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d800aa90 con 0x7f12e0155570 2024-04-22T06:32:12.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.657+0000 7f12e4db2640 1 -- 172.21.15.44:0/3419627954 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12e01e80b0 con 0x7f12e0155570 2024-04-22T06:32:12.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.657+0000 7f12e4db2640 1 -- 172.21.15.44:0/3419627954 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12e0157d70 con 0x7f12e0155570 2024-04-22T06:32:12.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.659+0000 7f12d6ffd640 1 -- 172.21.15.44:0/3419627954 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f12d800ac30 con 0x7f12e0155570 2024-04-22T06:32:12.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.660+0000 7f12d6ffd640 1 --2- 172.21.15.44:0/3419627954 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f12c006c8f0 0x7f12c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.660+0000 7f12d6ffd640 1 -- 172.21.15.44:0/3419627954 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f12d808bdf0 con 0x7f12e0155570 2024-04-22T06:32:12.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.660+0000 7f12de59c640 1 --2- 172.21.15.44:0/3419627954 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f12c006c8f0 0x7f12c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.660+0000 7f12e79a4740 1 Processor -- start 2024-04-22T06:32:12.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.661+0000 7f12de59c640 1 --2- 172.21.15.44:0/3419627954 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f12c006c8f0 0x7f12c006edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f12d0007c30 tx=0x7f12d0007b80 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:12.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.661+0000 7f12e79a4740 1 -- start start 2024-04-22T06:32:12.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.661+0000 7f12e79a4740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559e55fd8e40 0x559e55fe1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.661+0000 7f12e79a4740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e55f99ec0 0x559e55fe24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.661+0000 7f12e79a4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e55f95ca0 0x559e55fbdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.661+0000 7f12e79a4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559e55efcb10 con 0x559e55f95ca0 2024-04-22T06:32:12.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.661+0000 7f12e79a4740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x559e55e9a240 con 0x559e55f99ec0 2024-04-22T06:32:12.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.661+0000 7f12e79a4740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x559e55ef8ad0 con 0x559e55fd8e40 2024-04-22T06:32:12.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.661+0000 7f12ded9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e55f95ca0 0x559e55fbdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.661+0000 7f12ded9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e55f95ca0 0x559e55fbdee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36708/0 (socket says 172.21.15.44:36708) 2024-04-22T06:32:12.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.661+0000 7f12ded9d640 1 -- 172.21.15.44:0/3603221995 learned_addr learned my addr 172.21.15.44:0/3603221995 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:12.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.661+0000 7f12d7fff640 1 --2- 172.21.15.44:0/3603221995 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e55f99ec0 0x559e55fe24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.661+0000 7f12de59c640 1 --2- 172.21.15.44:0/3603221995 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559e55fd8e40 0x559e55fe1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.662+0000 7f12ded9d640 1 -- 172.21.15.44:0/3603221995 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559e55fd8e40 msgr2=0x559e55fe1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.662+0000 7f12ded9d640 1 --2- 172.21.15.44:0/3603221995 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559e55fd8e40 0x559e55fe1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.662+0000 7f12ded9d640 1 -- 172.21.15.44:0/3603221995 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e55f99ec0 msgr2=0x559e55fe24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.662+0000 7f12ded9d640 1 --2- 172.21.15.44:0/3603221995 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e55f99ec0 0x559e55fe24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.662+0000 7f12ded9d640 1 -- 172.21.15.44:0/3603221995 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e55c0f2d0 con 0x559e55f95ca0 2024-04-22T06:32:12.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.662+0000 7f12ded9d640 1 --2- 172.21.15.44:0/3603221995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e55f95ca0 0x559e55fbdee0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f12d800cfd0 tx=0x7f12d8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ba65335facff7d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:12.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.662+0000 7f12bffff640 1 -- 172.21.15.44:0/3603221995 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d8062300 con 0x559e55f95ca0 2024-04-22T06:32:12.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.662+0000 7f12bffff640 1 -- 172.21.15.44:0/3603221995 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12d8089070 con 0x559e55f95ca0 2024-04-22T06:32:12.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.662+0000 7f12bffff640 1 -- 172.21.15.44:0/3603221995 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d8057a40 con 0x559e55f95ca0 2024-04-22T06:32:12.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.663+0000 7f12e79a4740 1 -- 172.21.15.44:0/3603221995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e55f95ca0 msgr2=0x559e55fbdee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.663+0000 7f12e79a4740 1 --2- 172.21.15.44:0/3603221995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e55f95ca0 0x559e55fbdee0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f12d800cfd0 tx=0x7f12d8011000 comp rx=0 tx=0).stop 2024-04-22T06:32:12.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.663+0000 7f12e79a4740 1 -- 172.21.15.44:0/3603221995 shutdown_connections 2024-04-22T06:32:12.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.663+0000 7f12e79a4740 1 --2- 172.21.15.44:0/3603221995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e55f95ca0 0x559e55fbdee0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.663+0000 7f12e79a4740 1 --2- 172.21.15.44:0/3603221995 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e55f99ec0 0x559e55fe24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.663+0000 7f12e79a4740 1 --2- 172.21.15.44:0/3603221995 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559e55fd8e40 0x559e55fe1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.663+0000 7f12e79a4740 1 -- 172.21.15.44:0/3603221995 >> 172.21.15.44:0/3603221995 conn(0x559e55f94990 msgr2=0x559e55f97290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:12.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.663+0000 7f12e79a4740 1 -- 172.21.15.44:0/3603221995 shutdown_connections 2024-04-22T06:32:12.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.663+0000 7f12e79a4740 1 -- 172.21.15.44:0/3603221995 wait complete. 2024-04-22T06:32:12.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.663+0000 7f12e79a4740 1 -- start start 2024-04-22T06:32:12.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.663+0000 7f12e79a4740 1 Processor -- start 2024-04-22T06:32:12.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.664+0000 7f12e79a4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e55f99ec0 0x559e55fdd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.664+0000 7f12e79a4740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e55fd8e40 0x559e55fed0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.664+0000 7f12e79a4740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559e55fe03b0 0x559e55fed600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.664+0000 7f12e79a4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559e55f98250 con 0x559e55f99ec0 2024-04-22T06:32:12.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.664+0000 7f12e79a4740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x559e55e85190 con 0x559e55fd8e40 2024-04-22T06:32:12.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.664+0000 7f12e79a4740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x559e55f98530 con 0x559e55fe03b0 2024-04-22T06:32:12.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.664+0000 7f12e79a4740 10 client.0 _ll_register_callbacks cb 0x559e55e808b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T06:32:12.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.664+0000 7f12de59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e55f99ec0 0x559e55fdd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.664+0000 7f12de59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e55f99ec0 0x559e55fdd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36720/0 (socket says 172.21.15.44:36720) 2024-04-22T06:32:12.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.664+0000 7f12de59c640 1 -- 172.21.15.44:0/3113387681 learned_addr learned my addr 172.21.15.44:0/3113387681 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:12.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.664+0000 7f12d7fff640 1 --2- 172.21.15.44:0/3113387681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e55fd8e40 0x559e55fed0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.664+0000 7f12ded9d640 1 --2- 172.21.15.44:0/3113387681 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559e55fe03b0 0x559e55fed600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.665+0000 7f12de59c640 1 -- 172.21.15.44:0/3113387681 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559e55fe03b0 msgr2=0x559e55fed600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.665+0000 7f12de59c640 1 --2- 172.21.15.44:0/3113387681 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559e55fe03b0 0x559e55fed600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.665+0000 7f12de59c640 1 -- 172.21.15.44:0/3113387681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e55fd8e40 msgr2=0x559e55fed0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.665+0000 7f12de59c640 1 --2- 172.21.15.44:0/3113387681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e55fd8e40 0x559e55fed0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.665+0000 7f12de59c640 1 -- 172.21.15.44:0/3113387681 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e55cd96b0 con 0x559e55f99ec0 2024-04-22T06:32:12.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.665+0000 7f12d7fff640 1 --2- 172.21.15.44:0/3113387681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e55fd8e40 0x559e55fed0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:32:12.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.665+0000 7f12de59c640 1 --2- 172.21.15.44:0/3113387681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e55f99ec0 0x559e55fdd7b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f12d0014500 tx=0x7f12d0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7816cb5c60d5808 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:12.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.665+0000 7f12bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T06:32:12.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.665+0000 7f12bd7fa640 1 -- 172.21.15.44:0/3113387681 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d0005a60 con 0x559e55f99ec0 2024-04-22T06:32:12.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.665+0000 7f12bd7fa640 1 -- 172.21.15.44:0/3113387681 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f12d0005c00 con 0x559e55f99ec0 2024-04-22T06:32:12.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.665+0000 7f12e79a4740 1 -- 172.21.15.44:0/3113387681 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12d8089070 con 0x559e55f99ec0 2024-04-22T06:32:12.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.665+0000 7f12bd7fa640 1 -- 172.21.15.44:0/3113387681 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f12d0005e50 con 0x559e55f99ec0 2024-04-22T06:32:12.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.665+0000 7f12e79a4740 10 client.5247 fetch_fsmap learned FSMap version 18 2024-04-22T06:32:12.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.665+0000 7f12e79a4740 1 -- 172.21.15.44:0/3113387681 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x559e55fddcf0 con 0x559e55f99ec0 2024-04-22T06:32:12.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.665+0000 7f12bd7fa640 1 -- 172.21.15.44:0/3113387681 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12d8089070 con 0x559e55f99ec0 2024-04-22T06:32:12.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.665+0000 7f12bd7fa640 1 -- 172.21.15.44:0/3113387681 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 18) ==== 4792+0+0 (secure 0 0 0) 0x7f12d0020040 con 0x559e55f99ec0 2024-04-22T06:32:12.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.665+0000 7f12e79a4740 10 client.5247 fetch_fsmap finished waiting for FSMap version 18 2024-04-22T06:32:12.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.665+0000 7f12e79a4740 10 client.5247 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-22T06:32:12.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.665+0000 7f12e79a4740 20 client.5247 populate_metadata read hostname 'smithi044' 2024-04-22T06:32:12.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.665+0000 7f12e79a4740 1 --2- 172.21.15.44:0/3113387681 >> [v2:172.21.15.70:6834/924955785,v1:172.21.15.70:6838/924955785] conn(0x559e55fe0fd0 0x559e55ff8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.665+0000 7f12e79a4740 4 client.5247 mds_command: new command op to 4305 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T06:32:12.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.665+0000 7f12e79a4740 1 -- 172.21.15.44:0/3113387681 --> [v2:172.21.15.70:6834/924955785,v1:172.21.15.70:6838/924955785] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e55f0a530 con 0x559e55fe0fd0 2024-04-22T06:32:12.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.666+0000 7f12d7fff640 1 --2- 172.21.15.44:0/3113387681 >> [v2:172.21.15.70:6834/924955785,v1:172.21.15.70:6838/924955785] conn(0x559e55fe0fd0 0x559e55ff8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.666+0000 7f12d7fff640 1 --2- 172.21.15.44:0/3113387681 >> [v2:172.21.15.70:6834/924955785,v1:172.21.15.70:6838/924955785] conn(0x559e55fe0fd0 0x559e55ff8890 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=60eff48d4f82836f server_cookie=b26faf0cbadf04fb in_seq=0 out_seq=0 2024-04-22T06:32:12.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.666+0000 7f12bd7fa640 10 client.5247 ms_handle_connect on v2:172.21.15.70:6834/924955785 2024-04-22T06:32:12.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.667+0000 7f12bd7fa640 1 -- 172.21.15.44:0/3113387681 <== mds.1 v2:172.21.15.70:6834/924955785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559e55f0a530 con 0x559e55fe0fd0 2024-04-22T06:32:12.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.667+0000 7f12bd7fa640 10 client.5247 handle_command_reply: tid=0 2024-04-22T06:32:12.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.667+0000 7f12e79a4740 1 client.5247 shutdown 2024-04-22T06:32:12.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.667+0000 7f12e79a4740 1 -- 172.21.15.44:0/3113387681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e55f99ec0 msgr2=0x559e55fdd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.667+0000 7f12e79a4740 1 --2- 172.21.15.44:0/3113387681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e55f99ec0 0x559e55fdd7b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f12d0014500 tx=0x7f12d0000ed0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.667+0000 7f12e79a4740 1 -- 172.21.15.44:0/3113387681 shutdown_connections 2024-04-22T06:32:12.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.667+0000 7f12e79a4740 1 --2- 172.21.15.44:0/3113387681 >> [v2:172.21.15.70:6834/924955785,v1:172.21.15.70:6838/924955785] conn(0x559e55fe0fd0 0x559e55ff8890 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.667+0000 7f12e79a4740 1 --2- 172.21.15.44:0/3113387681 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559e55fe03b0 0x559e55fed600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.667+0000 7f12bd7fa640 0 client.5247 ms_handle_reset on v2:172.21.15.70:6834/924955785 2024-04-22T06:32:12.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.667+0000 7f12e79a4740 1 --2- 172.21.15.44:0/3113387681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e55fd8e40 0x559e55fed0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.667+0000 7f12e79a4740 1 --2- 172.21.15.44:0/3113387681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e55f99ec0 0x559e55fdd7b0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.667+0000 7f12e79a4740 1 -- 172.21.15.44:0/3113387681 >> 172.21.15.44:0/3113387681 conn(0x559e55f95ca0 msgr2=0x559e55fe1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:12.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.667+0000 7f12e79a4740 1 -- 172.21.15.44:0/3113387681 shutdown_connections 2024-04-22T06:32:12.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.667+0000 7f12e79a4740 1 -- 172.21.15.44:0/3113387681 wait complete. 2024-04-22T06:32:12.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.667+0000 7f12e79a4740 20 client.5247 trim_cache size 0 max 16384 2024-04-22T06:32:12.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.685+0000 7f12e79a4740 1 Processor -- start 2024-04-22T06:32:12.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.685+0000 7f12e79a4740 1 -- start start 2024-04-22T06:32:12.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.686+0000 7f12e79a4740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559e55f99ec0 0x559e55fde680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.686+0000 7f12e79a4740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e55fd8e40 0x559e55f920b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.686+0000 7f12e79a4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e55fe03b0 0x559e55fe1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.686+0000 7f12e79a4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f12e01f5200 con 0x559e55fe03b0 2024-04-22T06:32:12.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.686+0000 7f12e79a4740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x559e55f93030 con 0x559e55fd8e40 2024-04-22T06:32:12.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.686+0000 7f12e79a4740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x559e55f92ac0 con 0x559e55f99ec0 2024-04-22T06:32:12.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.686+0000 7f12ded9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e55fe03b0 0x559e55fe1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.686+0000 7f12ded9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e55fe03b0 0x559e55fe1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36724/0 (socket says 172.21.15.44:36724) 2024-04-22T06:32:12.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.686+0000 7f12ded9d640 1 -- 172.21.15.44:0/663232477 learned_addr learned my addr 172.21.15.44:0/663232477 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:12.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.686+0000 7f12ded9d640 1 -- 172.21.15.44:0/663232477 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559e55f99ec0 msgr2=0x559e55fde680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.686+0000 7f12ded9d640 1 --2- 172.21.15.44:0/663232477 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559e55f99ec0 0x559e55fde680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.686+0000 7f12ded9d640 1 -- 172.21.15.44:0/663232477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e55fd8e40 msgr2=0x559e55f920b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.686+0000 7f12d7fff640 1 --2- 172.21.15.44:0/663232477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e55fd8e40 0x559e55f920b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.687+0000 7f12ded9d640 1 --2- 172.21.15.44:0/663232477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e55fd8e40 0x559e55f920b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.687+0000 7f12ded9d640 1 -- 172.21.15.44:0/663232477 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e55fdf1d0 con 0x559e55fe03b0 2024-04-22T06:32:12.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.687+0000 7f12ded9d640 1 --2- 172.21.15.44:0/663232477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e55fe03b0 0x559e55fe1e60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f12d8089f70 tx=0x7f12d806f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=179a632c5360ca3a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:12.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.687+0000 7f12bf7fe640 1 -- 172.21.15.44:0/663232477 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d807e070 con 0x559e55fe03b0 2024-04-22T06:32:12.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.687+0000 7f12bf7fe640 1 -- 172.21.15.44:0/663232477 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12d8089040 con 0x559e55fe03b0 2024-04-22T06:32:12.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.687+0000 7f12bf7fe640 1 -- 172.21.15.44:0/663232477 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d80559e0 con 0x559e55fe03b0 2024-04-22T06:32:12.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.687+0000 7f12e79a4740 1 -- 172.21.15.44:0/663232477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e55fe03b0 msgr2=0x559e55fe1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.687+0000 7f12e79a4740 1 --2- 172.21.15.44:0/663232477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e55fe03b0 0x559e55fe1e60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f12d8089f70 tx=0x7f12d806f000 comp rx=0 tx=0).stop 2024-04-22T06:32:12.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.687+0000 7f12e79a4740 1 -- 172.21.15.44:0/663232477 shutdown_connections 2024-04-22T06:32:12.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.687+0000 7f12e79a4740 1 --2- 172.21.15.44:0/663232477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e55fe03b0 0x559e55fe1e60 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.687+0000 7f12e79a4740 1 --2- 172.21.15.44:0/663232477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e55fd8e40 0x559e55f920b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.687+0000 7f12e79a4740 1 --2- 172.21.15.44:0/663232477 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559e55f99ec0 0x559e55fde680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.687+0000 7f12e79a4740 1 -- 172.21.15.44:0/663232477 >> 172.21.15.44:0/663232477 conn(0x559e55f95ca0 msgr2=0x559e55f8d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:12.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.687+0000 7f12e79a4740 1 -- 172.21.15.44:0/663232477 shutdown_connections 2024-04-22T06:32:12.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.687+0000 7f12e79a4740 1 -- 172.21.15.44:0/663232477 wait complete. 2024-04-22T06:32:12.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.688+0000 7f12e79a4740 1 -- start start 2024-04-22T06:32:12.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.688+0000 7f12e79a4740 1 Processor -- start 2024-04-22T06:32:12.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.689+0000 7f12e79a4740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559e55f99ec0 0x559e55fe2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.689+0000 7f12e79a4740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e55fd8e40 0x559e55fb8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.689+0000 7f12e79a4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e55fe03b0 0x559e56050b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.689+0000 7f12e79a4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559e55fdf9b0 con 0x559e55fe03b0 2024-04-22T06:32:12.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.689+0000 7f12e79a4740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x559e55fddf00 con 0x559e55fd8e40 2024-04-22T06:32:12.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.689+0000 7f12e79a4740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x559e55fe1400 con 0x559e55f99ec0 2024-04-22T06:32:12.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.689+0000 7f12e79a4740 10 client.0 _ll_register_callbacks cb 0x7f12d0006ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T06:32:12.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.689+0000 7f12ded9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e55fe03b0 0x559e56050b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.689+0000 7f12ded9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e55fe03b0 0x559e56050b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36734/0 (socket says 172.21.15.44:36734) 2024-04-22T06:32:12.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.689+0000 7f12ded9d640 1 -- 172.21.15.44:0/4118319045 learned_addr learned my addr 172.21.15.44:0/4118319045 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:12.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.689+0000 7f12d7fff640 1 --2- 172.21.15.44:0/4118319045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e55fd8e40 0x559e55fb8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.689+0000 7f12de59c640 1 --2- 172.21.15.44:0/4118319045 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559e55f99ec0 0x559e55fe2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.689+0000 7f12ded9d640 1 -- 172.21.15.44:0/4118319045 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559e55f99ec0 msgr2=0x559e55fe2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.689+0000 7f12ded9d640 1 --2- 172.21.15.44:0/4118319045 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559e55f99ec0 0x559e55fe2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.689+0000 7f12ded9d640 1 -- 172.21.15.44:0/4118319045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e55fd8e40 msgr2=0x559e55fb8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.689+0000 7f12ded9d640 1 --2- 172.21.15.44:0/4118319045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e55fd8e40 0x559e55fb8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.689+0000 7f12ded9d640 1 -- 172.21.15.44:0/4118319045 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e55fb8e40 con 0x559e55fe03b0 2024-04-22T06:32:12.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.689+0000 7f12ded9d640 1 --2- 172.21.15.44:0/4118319045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e55fe03b0 0x559e56050b90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f12d80596d0 tx=0x7f12d8001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d4ad66ae35c7448 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:12.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.689+0000 7f12bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T06:32:12.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.690+0000 7f12bd7fa640 1 -- 172.21.15.44:0/4118319045 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d807e070 con 0x559e55fe03b0 2024-04-22T06:32:12.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.690+0000 7f12bd7fa640 1 -- 172.21.15.44:0/4118319045 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f12d8055db0 con 0x559e55fe03b0 2024-04-22T06:32:12.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.690+0000 7f12e79a4740 1 -- 172.21.15.44:0/4118319045 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12d8089040 con 0x559e55fe03b0 2024-04-22T06:32:12.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.690+0000 7f12bd7fa640 1 -- 172.21.15.44:0/4118319045 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f12d806a050 con 0x559e55fe03b0 2024-04-22T06:32:12.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.690+0000 7f12e79a4740 10 client.5253 fetch_fsmap learned FSMap version 18 2024-04-22T06:32:12.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.690+0000 7f12e79a4740 1 -- 172.21.15.44:0/4118319045 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x559e55fd95f0 con 0x559e55fe03b0 2024-04-22T06:32:12.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.690+0000 7f12bd7fa640 1 -- 172.21.15.44:0/4118319045 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12d8089040 con 0x559e55fe03b0 2024-04-22T06:32:12.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.690+0000 7f12bd7fa640 1 -- 172.21.15.44:0/4118319045 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 18) ==== 4792+0+0 (secure 0 0 0) 0x7f12d806d020 con 0x559e55fe03b0 2024-04-22T06:32:12.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.690+0000 7f12e79a4740 10 client.5253 fetch_fsmap finished waiting for FSMap version 18 2024-04-22T06:32:12.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.690+0000 7f12e79a4740 10 client.5253 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-22T06:32:12.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.690+0000 7f12e79a4740 20 client.5253 populate_metadata read hostname 'smithi044' 2024-04-22T06:32:12.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.690+0000 7f12e79a4740 1 --2- 172.21.15.44:0/4118319045 >> [v2:172.21.15.70:6834/924955785,v1:172.21.15.70:6838/924955785] conn(0x559e55fcf210 0x559e55fcf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.690+0000 7f12e79a4740 4 client.5253 mds_command: new command op to 4305 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T06:32:12.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.690+0000 7f12e79a4740 1 -- 172.21.15.44:0/4118319045 --> [v2:172.21.15.70:6834/924955785,v1:172.21.15.70:6838/924955785] -- command(tid 0: {"prefix": "status"}) -- 0x559e55f043a0 con 0x559e55fcf210 2024-04-22T06:32:12.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.690+0000 7f12de59c640 1 --2- 172.21.15.44:0/4118319045 >> [v2:172.21.15.70:6834/924955785,v1:172.21.15.70:6838/924955785] conn(0x559e55fcf210 0x559e55fcf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.691+0000 7f12bd7fa640 10 client.5253 ms_handle_connect on v2:172.21.15.70:6834/924955785 2024-04-22T06:32:12.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.691+0000 7f12de59c640 1 --2- 172.21.15.44:0/4118319045 >> [v2:172.21.15.70:6834/924955785,v1:172.21.15.70:6838/924955785] conn(0x559e55fcf210 0x559e55fcf5f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c5608606727f6c54 server_cookie=1645a15711b080a5 in_seq=0 out_seq=0 2024-04-22T06:32:12.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.691+0000 7f12bd7fa640 1 -- 172.21.15.44:0/4118319045 <== mds.1 v2:172.21.15.70:6834/924955785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x559e55f043a0 con 0x559e55fcf210 2024-04-22T06:32:12.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.691+0000 7f12bd7fa640 10 client.5253 handle_command_reply: tid=0 2024-04-22T06:32:12.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.691+0000 7f12e79a4740 1 client.5253 shutdown 2024-04-22T06:32:12.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.691+0000 7f12e79a4740 1 -- 172.21.15.44:0/4118319045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e55fe03b0 msgr2=0x559e56050b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.691+0000 7f12e79a4740 1 --2- 172.21.15.44:0/4118319045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e55fe03b0 0x559e56050b90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f12d80596d0 tx=0x7f12d8001250 comp rx=0 tx=0).stop 2024-04-22T06:32:12.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.691+0000 7f12e79a4740 1 -- 172.21.15.44:0/4118319045 shutdown_connections 2024-04-22T06:32:12.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.691+0000 7f12e79a4740 1 --2- 172.21.15.44:0/4118319045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e55fe03b0 0x559e56050b90 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.691+0000 7f12e79a4740 1 --2- 172.21.15.44:0/4118319045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559e55fd8e40 0x559e55fb8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.691+0000 7f12e79a4740 1 --2- 172.21.15.44:0/4118319045 >> [v2:172.21.15.70:6834/924955785,v1:172.21.15.70:6838/924955785] conn(0x559e55fcf210 0x559e55fcf5f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.691+0000 7f12e79a4740 1 --2- 172.21.15.44:0/4118319045 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559e55f99ec0 0x559e55fe2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.691+0000 7f12bd7fa640 0 client.5253 ms_handle_reset on v2:172.21.15.70:6834/924955785 2024-04-22T06:32:12.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.691+0000 7f12e79a4740 1 -- 172.21.15.44:0/4118319045 >> 172.21.15.44:0/4118319045 conn(0x559e55f95ca0 msgr2=0x559e55f8d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:12.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.691+0000 7f12e79a4740 1 -- 172.21.15.44:0/4118319045 shutdown_connections 2024-04-22T06:32:12.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.691+0000 7f12e79a4740 1 -- 172.21.15.44:0/4118319045 wait complete. 2024-04-22T06:32:12.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.691+0000 7f12e79a4740 20 client.5253 trim_cache size 0 max 16384 2024-04-22T06:32:12.698 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-22T06:32:12.698 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "9b0a34ca-403d-4513-8a66-d3749ba559aa", 2024-04-22T06:32:12.698 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 1, 2024-04-22T06:32:12.698 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 4305, 2024-04-22T06:32:12.698 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-04-22T06:32:12.698 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-04-22T06:32:12.698 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-04-22T06:32:12.698 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 40.610993948999997, 2024-04-22T06:32:12.699 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 18, 2024-04-22T06:32:12.699 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 27, 2024-04-22T06:32:12.699 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 27, 2024-04-22T06:32:12.699 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 72.584420445999996 2024-04-22T06:32:12.699 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-22T06:32:12.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.692+0000 7f12e4db2640 1 -- 172.21.15.44:0/3419627954 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f12c006c8f0 msgr2=0x7f12c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.692+0000 7f12e4db2640 1 --2- 172.21.15.44:0/3419627954 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f12c006c8f0 0x7f12c006edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f12d0007c30 tx=0x7f12d0007b80 comp rx=0 tx=0).stop 2024-04-22T06:32:12.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.692+0000 7f12e4db2640 1 -- 172.21.15.44:0/3419627954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12e0155570 msgr2=0x7f12e01e6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.692+0000 7f12e4db2640 1 --2- 172.21.15.44:0/3419627954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12e0155570 0x7f12e01e6110 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f12d800c7a0 tx=0x7f12d800cc60 comp rx=0 tx=0).stop 2024-04-22T06:32:12.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.692+0000 7f12e4db2640 1 -- 172.21.15.44:0/3419627954 shutdown_connections 2024-04-22T06:32:12.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.692+0000 7f12e4db2640 1 --2- 172.21.15.44:0/3419627954 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f12c006c8f0 0x7f12c006edb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.692+0000 7f12e4db2640 1 --2- 172.21.15.44:0/3419627954 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f12e00cc4a0 0x7f12e01e58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.692+0000 7f12e4db2640 1 --2- 172.21.15.44:0/3419627954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12e0155570 0x7f12e01e6110 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.692+0000 7f12e4db2640 1 --2- 172.21.15.44:0/3419627954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12e006cd30 0x7f12e0158290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.692+0000 7f12e4db2640 1 -- 172.21.15.44:0/3419627954 >> 172.21.15.44:0/3419627954 conn(0x7f12e0075870 msgr2=0x7f12e01598d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:12.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.692+0000 7f12e4db2640 1 -- 172.21.15.44:0/3419627954 shutdown_connections 2024-04-22T06:32:12.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.692+0000 7f12e4db2640 1 -- 172.21.15.44:0/3419627954 wait complete. 2024-04-22T06:32:12.702 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-22T06:32:12.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.805+0000 7f0f0d523640 1 Processor -- start 2024-04-22T06:32:12.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.805+0000 7f0f0d523640 1 -- start start 2024-04-22T06:32:12.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.806+0000 7f0f0d523640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0f080cc8e0 0x7f0f08166ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.806+0000 7f0f0d523640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f080cc0a0 con 0x7f0f08155570 2024-04-22T06:32:12.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.806+0000 7f0f0d523640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f0815b650 con 0x7f0f080cc4a0 2024-04-22T06:32:12.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.806+0000 7f0f0d523640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f0f0806c8a0 con 0x7f0f080cc8e0 2024-04-22T06:32:12.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.806+0000 7f0f06ffd640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f0f08155570 0x7f0f080cb990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:55546/0 (socket says 172.21.15.44:55546) 2024-04-22T06:32:12.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.806+0000 7f0f06ffd640 1 -- 172.21.15.44:0/493459061 learned_addr learned my addr 172.21.15.44:0/493459061 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:32:12.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.806+0000 7f0f077fe640 1 --2- 172.21.15.44:0/493459061 >> v2:172.21.15.70:3300/0 conn(0x7f0f080cc8e0 0x7f0f08166ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.806+0000 7f0f05ffb640 1 -- 172.21.15.44:0/493459061 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 341925501 0 0) 0x7f0f00001110 con 0x7f0f08155570 2024-04-22T06:32:12.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.806+0000 7f0f05ffb640 1 -- 172.21.15.44:0/493459061 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ef4003520 con 0x7f0f08155570 2024-04-22T06:32:12.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.806+0000 7f0f077fe640 1 -- 172.21.15.44:0/493459061 >> v1:172.21.15.70:6789/0 conn(0x7f0f080cc4a0 legacy=0x7f0f0806cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.806+0000 7f0f077fe640 1 -- 172.21.15.44:0/493459061 >> v1:172.21.15.44:6789/0 conn(0x7f0f08155570 legacy=0x7f0f080cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.806+0000 7f0f077fe640 1 -- 172.21.15.44:0/493459061 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f0815b930 con 0x7f0f080cc8e0 2024-04-22T06:32:12.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.807+0000 7f0f077fe640 1 --2- 172.21.15.44:0/493459061 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f080cc8e0 0x7f0f08166ae0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8009d90 tx=0x7f0ef802beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ad8e715a0f526ae server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:12.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.807+0000 7f0f05ffb640 1 -- 172.21.15.44:0/493459061 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8039070 con 0x7f0f080cc8e0 2024-04-22T06:32:12.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.807+0000 7f0f05ffb640 1 -- 172.21.15.44:0/493459061 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ef802cc60 con 0x7f0f080cc8e0 2024-04-22T06:32:12.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.807+0000 7f0f05ffb640 1 -- 172.21.15.44:0/493459061 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8034660 con 0x7f0f080cc8e0 2024-04-22T06:32:12.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.808+0000 7f0f0d523640 1 -- 172.21.15.44:0/493459061 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f080cc8e0 msgr2=0x7f0f08166ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.808+0000 7f0f0d523640 1 --2- 172.21.15.44:0/493459061 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f080cc8e0 0x7f0f08166ae0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8009d90 tx=0x7f0ef802beb0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.808+0000 7f0f0d523640 1 -- 172.21.15.44:0/493459061 shutdown_connections 2024-04-22T06:32:12.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.808+0000 7f0f0d523640 1 --2- 172.21.15.44:0/493459061 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f080cc8e0 0x7f0f08166ae0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.808+0000 7f0f0d523640 1 -- 172.21.15.44:0/493459061 >> 172.21.15.44:0/493459061 conn(0x7f0f08075870 msgr2=0x7f0f08073020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:12.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.808+0000 7f0f0d523640 1 -- 172.21.15.44:0/493459061 shutdown_connections 2024-04-22T06:32:12.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.808+0000 7f0f0d523640 1 -- 172.21.15.44:0/493459061 wait complete. 2024-04-22T06:32:12.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.809+0000 7f0f0d523640 1 Processor -- start 2024-04-22T06:32:12.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.809+0000 7f0f0d523640 1 -- start start 2024-04-22T06:32:12.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.810+0000 7f0f0d523640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f080cc4a0 0x7f0f080c6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.810+0000 7f0f0d523640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f080cc8e0 0x7f0f080cd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.810+0000 7f0f0d523640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f08155570 0x7f0f080cd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.810+0000 7f0f0d523640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0815bef0 con 0x7f0f08155570 2024-04-22T06:32:12.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.810+0000 7f0f0d523640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0815c1d0 con 0x7f0f080cc8e0 2024-04-22T06:32:12.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.810+0000 7f0f0d523640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f0f080cccf0 con 0x7f0f080cc4a0 2024-04-22T06:32:12.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.810+0000 7f0f077fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f08155570 0x7f0f080cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.810+0000 7f0f077fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f08155570 0x7f0f080cd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36746/0 (socket says 172.21.15.44:36746) 2024-04-22T06:32:12.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.810+0000 7f0f077fe640 1 -- 172.21.15.44:0/2097051416 learned_addr learned my addr 172.21.15.44:0/2097051416 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:12.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.810+0000 7f0f06ffd640 1 --2- 172.21.15.44:0/2097051416 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f080cc4a0 0x7f0f080c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.810+0000 7f0f067fc640 1 --2- 172.21.15.44:0/2097051416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f080cc8e0 0x7f0f080cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.810+0000 7f0f077fe640 1 -- 172.21.15.44:0/2097051416 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f080cc4a0 msgr2=0x7f0f080c6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.810+0000 7f0f077fe640 1 --2- 172.21.15.44:0/2097051416 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f080cc4a0 0x7f0f080c6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.810+0000 7f0f077fe640 1 -- 172.21.15.44:0/2097051416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f080cc8e0 msgr2=0x7f0f080cd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.810+0000 7f0f077fe640 1 --2- 172.21.15.44:0/2097051416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f080cc8e0 0x7f0f080cd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.810+0000 7f0f077fe640 1 -- 172.21.15.44:0/2097051416 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f080c68a0 con 0x7f0f08155570 2024-04-22T06:32:12.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.810+0000 7f0f067fc640 1 --2- 172.21.15.44:0/2097051416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f080cc8e0 0x7f0f080cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:32:12.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.811+0000 7f0f077fe640 1 --2- 172.21.15.44:0/2097051416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f08155570 0x7f0f080cd880 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8009d90 tx=0x7f0ef8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:12.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.811+0000 7f0eff7fe640 1 -- 172.21.15.44:0/2097051416 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8039050 con 0x7f0f08155570 2024-04-22T06:32:12.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.811+0000 7f0eff7fe640 1 -- 172.21.15.44:0/2097051416 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ef8045070 con 0x7f0f08155570 2024-04-22T06:32:12.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.811+0000 7f0f0d523640 1 -- 172.21.15.44:0/2097051416 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f08158310 con 0x7f0f08155570 2024-04-22T06:32:12.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.811+0000 7f0eff7fe640 1 -- 172.21.15.44:0/2097051416 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8009450 con 0x7f0f08155570 2024-04-22T06:32:12.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.811+0000 7f0f0d523640 1 -- 172.21.15.44:0/2097051416 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f081f4ba0 con 0x7f0f08155570 2024-04-22T06:32:12.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.812+0000 7f0eff7fe640 1 -- 172.21.15.44:0/2097051416 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ef803e850 con 0x7f0f08155570 2024-04-22T06:32:12.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.813+0000 7f0eff7fe640 1 --2- 172.21.15.44:0/2097051416 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f0ee806c8f0 0x7f0ee806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.813+0000 7f0eff7fe640 1 -- 172.21.15.44:0/2097051416 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f0ef80aed90 con 0x7f0f08155570 2024-04-22T06:32:12.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.813+0000 7f0f06ffd640 1 --2- 172.21.15.44:0/2097051416 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f0ee806c8f0 0x7f0ee806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.813+0000 7f0f06ffd640 1 --2- 172.21.15.44:0/2097051416 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f0ee806c8f0 0x7f0ee806edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0f000012b0 tx=0x7f0f00009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:12.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.814+0000 7f0f101eb740 1 Processor -- start 2024-04-22T06:32:12.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.814+0000 7f0f101eb740 1 -- start start 2024-04-22T06:32:12.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.815+0000 7f0f101eb740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d66f4de40 0x560d66f56f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.815+0000 7f0f101eb740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560d66f0eec0 0x560d66f574a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.815+0000 7f0f101eb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d66f0aca0 0x560d66f32ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.815+0000 7f0f101eb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x560d66e71b10 con 0x560d66f0aca0 2024-04-22T06:32:12.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.815+0000 7f0f101eb740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x560d66e0f240 con 0x560d66f4de40 2024-04-22T06:32:12.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.815+0000 7f0f101eb740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x560d66e6dad0 con 0x560d66f0eec0 2024-04-22T06:32:12.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.815+0000 7f0f077fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d66f0aca0 0x560d66f32ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.815+0000 7f0f077fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d66f0aca0 0x560d66f32ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36760/0 (socket says 172.21.15.44:36760) 2024-04-22T06:32:12.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.815+0000 7f0f077fe640 1 -- 172.21.15.44:0/2006067667 learned_addr learned my addr 172.21.15.44:0/2006067667 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:12.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.815+0000 7f0f06ffd640 1 --2- 172.21.15.44:0/2006067667 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d66f4de40 0x560d66f56f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.815+0000 7f0f067fc640 1 --2- 172.21.15.44:0/2006067667 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560d66f0eec0 0x560d66f574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.815+0000 7f0f077fe640 1 -- 172.21.15.44:0/2006067667 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560d66f0eec0 msgr2=0x560d66f574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.815+0000 7f0f077fe640 1 --2- 172.21.15.44:0/2006067667 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560d66f0eec0 0x560d66f574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.815+0000 7f0f077fe640 1 -- 172.21.15.44:0/2006067667 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d66f4de40 msgr2=0x560d66f56f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.815+0000 7f0f077fe640 1 --2- 172.21.15.44:0/2006067667 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d66f4de40 0x560d66f56f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.815+0000 7f0f077fe640 1 -- 172.21.15.44:0/2006067667 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d66b842d0 con 0x560d66f0aca0 2024-04-22T06:32:12.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.815+0000 7f0f077fe640 1 --2- 172.21.15.44:0/2006067667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d66f0aca0 0x560d66f32ee0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8034fd0 tx=0x7f0ef807c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54659a1b633bfb9e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:12.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.815+0000 7f0efcff9640 1 -- 172.21.15.44:0/2006067667 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef807ce40 con 0x560d66f0aca0 2024-04-22T06:32:12.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.815+0000 7f0efcff9640 1 -- 172.21.15.44:0/2006067667 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ef803a040 con 0x560d66f0aca0 2024-04-22T06:32:12.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.815+0000 7f0efcff9640 1 -- 172.21.15.44:0/2006067667 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef807c740 con 0x560d66f0aca0 2024-04-22T06:32:12.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.816+0000 7f0f101eb740 1 -- 172.21.15.44:0/2006067667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d66f0aca0 msgr2=0x560d66f32ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.816+0000 7f0f101eb740 1 --2- 172.21.15.44:0/2006067667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d66f0aca0 0x560d66f32ee0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8034fd0 tx=0x7f0ef807c3b0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.816+0000 7f0f101eb740 1 -- 172.21.15.44:0/2006067667 shutdown_connections 2024-04-22T06:32:12.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.816+0000 7f0f101eb740 1 --2- 172.21.15.44:0/2006067667 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560d66f0eec0 0x560d66f574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.816+0000 7f0f101eb740 1 --2- 172.21.15.44:0/2006067667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d66f0aca0 0x560d66f32ee0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.816+0000 7f0f101eb740 1 --2- 172.21.15.44:0/2006067667 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d66f4de40 0x560d66f56f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.816+0000 7f0f101eb740 1 -- 172.21.15.44:0/2006067667 >> 172.21.15.44:0/2006067667 conn(0x560d66f09990 msgr2=0x560d66f0c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:12.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.816+0000 7f0f101eb740 1 -- 172.21.15.44:0/2006067667 shutdown_connections 2024-04-22T06:32:12.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.816+0000 7f0f101eb740 1 -- 172.21.15.44:0/2006067667 wait complete. 2024-04-22T06:32:12.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.817+0000 7f0f101eb740 1 -- start start 2024-04-22T06:32:12.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.817+0000 7f0f101eb740 1 Processor -- start 2024-04-22T06:32:12.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.817+0000 7f0f101eb740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d66f0eec0 0x560d66f2d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.817+0000 7f0f101eb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d66f4de40 0x560d66f62410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.817+0000 7f0f101eb740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560d66f541f0 0x560d66f475f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.817+0000 7f0f101eb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x560d66e71b10 con 0x560d66f4de40 2024-04-22T06:32:12.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.817+0000 7f0f101eb740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x560d66dfa190 con 0x560d66f0eec0 2024-04-22T06:32:12.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.817+0000 7f0f101eb740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x560d66f0d250 con 0x560d66f541f0 2024-04-22T06:32:12.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.817+0000 7f0f101eb740 10 client.0 _ll_register_callbacks cb 0x560d66df58b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T06:32:12.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.818+0000 7f0f067fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d66f4de40 0x560d66f62410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.818+0000 7f0f067fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d66f4de40 0x560d66f62410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36772/0 (socket says 172.21.15.44:36772) 2024-04-22T06:32:12.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.818+0000 7f0f067fc640 1 -- 172.21.15.44:0/622983105 learned_addr learned my addr 172.21.15.44:0/622983105 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:12.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.818+0000 7f0f06ffd640 1 --2- 172.21.15.44:0/622983105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d66f0eec0 0x560d66f2d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.818+0000 7f0f077fe640 1 --2- 172.21.15.44:0/622983105 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560d66f541f0 0x560d66f475f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.818+0000 7f0f067fc640 1 -- 172.21.15.44:0/622983105 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560d66f541f0 msgr2=0x560d66f475f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.818+0000 7f0f067fc640 1 --2- 172.21.15.44:0/622983105 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560d66f541f0 0x560d66f475f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.818+0000 7f0f067fc640 1 -- 172.21.15.44:0/622983105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d66f0eec0 msgr2=0x560d66f2d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.818+0000 7f0f067fc640 1 --2- 172.21.15.44:0/622983105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d66f0eec0 0x560d66f2d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.818+0000 7f0f067fc640 1 -- 172.21.15.44:0/622983105 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d66c4e6b0 con 0x560d66f4de40 2024-04-22T06:32:12.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.818+0000 7f0f067fc640 1 --2- 172.21.15.44:0/622983105 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d66f4de40 0x560d66f62410 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0001e80 tx=0x7f0ef00116e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4844ce809990c35f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:12.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.818+0000 7f0ee5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T06:32:12.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.818+0000 7f0ee5ffb640 1 -- 172.21.15.44:0/622983105 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef00052e0 con 0x560d66f4de40 2024-04-22T06:32:12.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.818+0000 7f0ee5ffb640 1 -- 172.21.15.44:0/622983105 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f0ef00035f0 con 0x560d66f4de40 2024-04-22T06:32:12.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.818+0000 7f0f101eb740 1 -- 172.21.15.44:0/622983105 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ef803a040 con 0x560d66f4de40 2024-04-22T06:32:12.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.818+0000 7f0ee5ffb640 1 -- 172.21.15.44:0/622983105 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f0ef0022050 con 0x560d66f4de40 2024-04-22T06:32:12.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.818+0000 7f0f101eb740 10 client.5265 fetch_fsmap learned FSMap version 18 2024-04-22T06:32:12.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.818+0000 7f0f101eb740 1 -- 172.21.15.44:0/622983105 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x560d66f62950 con 0x560d66f4de40 2024-04-22T06:32:12.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.819+0000 7f0ee5ffb640 1 -- 172.21.15.44:0/622983105 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ef803a040 con 0x560d66f4de40 2024-04-22T06:32:12.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.819+0000 7f0ee5ffb640 1 -- 172.21.15.44:0/622983105 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 18) ==== 4792+0+0 (secure 0 0 0) 0x7f0ef0025020 con 0x560d66f4de40 2024-04-22T06:32:12.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.819+0000 7f0f101eb740 10 client.5265 fetch_fsmap finished waiting for FSMap version 18 2024-04-22T06:32:12.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.819+0000 7f0f101eb740 10 client.5265 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-22T06:32:12.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.819+0000 7f0f101eb740 20 client.5265 populate_metadata read hostname 'smithi044' 2024-04-22T06:32:12.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.819+0000 7f0f101eb740 1 --2- 172.21.15.44:0/622983105 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x560d66f529a0 0x560d66f6d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.819+0000 7f0f101eb740 4 client.5265 mds_command: new command op to 4308 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T06:32:12.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.819+0000 7f0f101eb740 1 -- 172.21.15.44:0/622983105 --> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d66e7f530 con 0x560d66f529a0 2024-04-22T06:32:12.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.819+0000 7f0f06ffd640 1 --2- 172.21.15.44:0/622983105 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x560d66f529a0 0x560d66f6d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.819+0000 7f0f06ffd640 1 --2- 172.21.15.44:0/622983105 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x560d66f529a0 0x560d66f6d8d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2beef083f825c285 server_cookie=2d794f8738c3c3f7 in_seq=0 out_seq=0 2024-04-22T06:32:12.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.819+0000 7f0ee5ffb640 10 client.5265 ms_handle_connect on v2:172.21.15.70:6837/2915150040 2024-04-22T06:32:12.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.820+0000 7f0ee5ffb640 1 -- 172.21.15.44:0/622983105 <== mds.0 v2:172.21.15.70:6837/2915150040 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560d66e7f530 con 0x560d66f529a0 2024-04-22T06:32:12.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.820+0000 7f0ee5ffb640 10 client.5265 handle_command_reply: tid=0 2024-04-22T06:32:12.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.820+0000 7f0f101eb740 1 client.5265 shutdown 2024-04-22T06:32:12.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.820+0000 7f0f101eb740 1 -- 172.21.15.44:0/622983105 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d66f4de40 msgr2=0x560d66f62410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.820+0000 7f0f101eb740 1 --2- 172.21.15.44:0/622983105 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d66f4de40 0x560d66f62410 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0001e80 tx=0x7f0ef00116e0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.820+0000 7f0f101eb740 1 -- 172.21.15.44:0/622983105 shutdown_connections 2024-04-22T06:32:12.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.820+0000 7f0f101eb740 1 --2- 172.21.15.44:0/622983105 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560d66f541f0 0x560d66f475f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.820+0000 7f0f101eb740 1 --2- 172.21.15.44:0/622983105 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x560d66f529a0 0x560d66f6d8d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.820+0000 7f0f101eb740 1 --2- 172.21.15.44:0/622983105 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d66f4de40 0x560d66f62410 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.820+0000 7f0f101eb740 1 --2- 172.21.15.44:0/622983105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d66f0eec0 0x560d66f2d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.820+0000 7f0ee5ffb640 0 client.5265 ms_handle_reset on v2:172.21.15.70:6837/2915150040 2024-04-22T06:32:12.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.820+0000 7f0f101eb740 1 -- 172.21.15.44:0/622983105 >> 172.21.15.44:0/622983105 conn(0x560d66f0aca0 msgr2=0x560d66f0da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:12.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.820+0000 7f0f101eb740 1 -- 172.21.15.44:0/622983105 shutdown_connections 2024-04-22T06:32:12.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.820+0000 7f0f101eb740 1 -- 172.21.15.44:0/622983105 wait complete. 2024-04-22T06:32:12.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.820+0000 7f0f101eb740 20 client.5265 trim_cache size 0 max 16384 2024-04-22T06:32:12.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.839+0000 7f0f101eb740 1 Processor -- start 2024-04-22T06:32:12.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.839+0000 7f0f101eb740 1 -- start start 2024-04-22T06:32:12.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.839+0000 7f0f101eb740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560d66f0eec0 0x560d66f56f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.839+0000 7f0f101eb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d66f4de40 0x560d66f72900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.839+0000 7f0f101eb740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d66f541f0 0x560d66f72e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.839+0000 7f0f101eb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0f08157550 con 0x560d66f4de40 2024-04-22T06:32:12.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.839+0000 7f0f101eb740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x560d66f53d60 con 0x560d66f541f0 2024-04-22T06:32:12.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.839+0000 7f0f101eb740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x560d66f61ef0 con 0x560d66f0eec0 2024-04-22T06:32:12.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.839+0000 7f0f067fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d66f4de40 0x560d66f72900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.839+0000 7f0f067fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d66f4de40 0x560d66f72900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36786/0 (socket says 172.21.15.44:36786) 2024-04-22T06:32:12.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.839+0000 7f0f067fc640 1 -- 172.21.15.44:0/2190394439 learned_addr learned my addr 172.21.15.44:0/2190394439 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:12.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.840+0000 7f0f067fc640 1 -- 172.21.15.44:0/2190394439 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560d66f0eec0 msgr2=0x560d66f56f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.840+0000 7f0f067fc640 1 --2- 172.21.15.44:0/2190394439 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560d66f0eec0 0x560d66f56f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.840+0000 7f0f067fc640 1 -- 172.21.15.44:0/2190394439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d66f541f0 msgr2=0x560d66f72e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.840+0000 7f0f067fc640 1 --2- 172.21.15.44:0/2190394439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d66f541f0 0x560d66f72e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.840+0000 7f0f067fc640 1 -- 172.21.15.44:0/2190394439 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d66f09e60 con 0x560d66f4de40 2024-04-22T06:32:12.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.840+0000 7f0f067fc640 1 --2- 172.21.15.44:0/2190394439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d66f4de40 0x560d66f72900 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0011c10 tx=0x7f0ef000c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8892b9d00f817dc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:12.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.840+0000 7f0ee7fff640 1 -- 172.21.15.44:0/2190394439 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef0001be0 con 0x560d66f4de40 2024-04-22T06:32:12.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.840+0000 7f0ee7fff640 1 -- 172.21.15.44:0/2190394439 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ef001a070 con 0x560d66f4de40 2024-04-22T06:32:12.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.840+0000 7f0ee7fff640 1 -- 172.21.15.44:0/2190394439 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef00035f0 con 0x560d66f4de40 2024-04-22T06:32:12.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.840+0000 7f0f101eb740 1 -- 172.21.15.44:0/2190394439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d66f4de40 msgr2=0x560d66f72900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.840+0000 7f0f101eb740 1 --2- 172.21.15.44:0/2190394439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d66f4de40 0x560d66f72900 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0011c10 tx=0x7f0ef000c000 comp rx=0 tx=0).stop 2024-04-22T06:32:12.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.841+0000 7f0f101eb740 1 -- 172.21.15.44:0/2190394439 shutdown_connections 2024-04-22T06:32:12.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.841+0000 7f0f101eb740 1 --2- 172.21.15.44:0/2190394439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d66f541f0 0x560d66f72e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.841+0000 7f0f101eb740 1 --2- 172.21.15.44:0/2190394439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d66f4de40 0x560d66f72900 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.841+0000 7f0f101eb740 1 --2- 172.21.15.44:0/2190394439 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560d66f0eec0 0x560d66f56f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.841+0000 7f0f101eb740 1 -- 172.21.15.44:0/2190394439 >> 172.21.15.44:0/2190394439 conn(0x560d66f0aca0 msgr2=0x560d66f473c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:12.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.841+0000 7f0f101eb740 1 -- 172.21.15.44:0/2190394439 shutdown_connections 2024-04-22T06:32:12.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.841+0000 7f0f101eb740 1 -- 172.21.15.44:0/2190394439 wait complete. 2024-04-22T06:32:12.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.841+0000 7f0f101eb740 1 -- start start 2024-04-22T06:32:12.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.841+0000 7f0f101eb740 1 Processor -- start 2024-04-22T06:32:12.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.842+0000 7f0f101eb740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d66f0eec0 0x560d66fc3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.842+0000 7f0f101eb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d66f4de40 0x560d66f29370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.842+0000 7f0f101eb740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560d66f541f0 0x560d66f298b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.842+0000 7f0f101eb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x560d66f08780 con 0x560d66f4de40 2024-04-22T06:32:12.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.842+0000 7f0f101eb740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x560d66f2df90 con 0x560d66f0eec0 2024-04-22T06:32:12.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.842+0000 7f0f101eb740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x560d66f52e20 con 0x560d66f541f0 2024-04-22T06:32:12.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.842+0000 7f0f101eb740 10 client.0 _ll_register_callbacks cb 0x7f0ef0001ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T06:32:12.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.842+0000 7f0f067fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d66f4de40 0x560d66f29370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.842+0000 7f0f067fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d66f4de40 0x560d66f29370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36798/0 (socket says 172.21.15.44:36798) 2024-04-22T06:32:12.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.842+0000 7f0f067fc640 1 -- 172.21.15.44:0/3324279107 learned_addr learned my addr 172.21.15.44:0/3324279107 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:12.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.842+0000 7f0f06ffd640 1 --2- 172.21.15.44:0/3324279107 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d66f0eec0 0x560d66fc3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.842+0000 7f0f077fe640 1 --2- 172.21.15.44:0/3324279107 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560d66f541f0 0x560d66f298b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.842+0000 7f0f067fc640 1 -- 172.21.15.44:0/3324279107 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560d66f541f0 msgr2=0x560d66f298b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.842+0000 7f0f067fc640 1 --2- 172.21.15.44:0/3324279107 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560d66f541f0 0x560d66f298b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.842+0000 7f0f067fc640 1 -- 172.21.15.44:0/3324279107 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d66f0eec0 msgr2=0x560d66fc3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:12.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.842+0000 7f0f067fc640 1 --2- 172.21.15.44:0/3324279107 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d66f0eec0 0x560d66fc3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.842+0000 7f0f067fc640 1 -- 172.21.15.44:0/3324279107 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d66f50eb0 con 0x560d66f4de40 2024-04-22T06:32:12.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.843+0000 7f0f067fc640 1 --2- 172.21.15.44:0/3324279107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d66f4de40 0x560d66f29370 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0001140 tx=0x7f0ef0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fbbb130f7ee0fd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:12.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.843+0000 7f0ee5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T06:32:12.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.843+0000 7f0ee5ffb640 1 -- 172.21.15.44:0/3324279107 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef0034060 con 0x560d66f4de40 2024-04-22T06:32:12.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.843+0000 7f0ee5ffb640 1 -- 172.21.15.44:0/3324279107 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f0ef0010040 con 0x560d66f4de40 2024-04-22T06:32:12.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.843+0000 7f0f101eb740 1 -- 172.21.15.44:0/3324279107 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ef001a070 con 0x560d66f4de40 2024-04-22T06:32:12.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.843+0000 7f0ee5ffb640 1 -- 172.21.15.44:0/3324279107 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f0ef003a050 con 0x560d66f4de40 2024-04-22T06:32:12.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.843+0000 7f0f101eb740 10 client.5271 fetch_fsmap learned FSMap version 18 2024-04-22T06:32:12.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.843+0000 7f0f101eb740 1 -- 172.21.15.44:0/3324279107 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x560d66f2c510 con 0x560d66f4de40 2024-04-22T06:32:12.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.843+0000 7f0ee5ffb640 1 -- 172.21.15.44:0/3324279107 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ef001a070 con 0x560d66f4de40 2024-04-22T06:32:12.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.843+0000 7f0ee5ffb640 1 -- 172.21.15.44:0/3324279107 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 18) ==== 4792+0+0 (secure 0 0 0) 0x7f0ef003d020 con 0x560d66f4de40 2024-04-22T06:32:12.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.843+0000 7f0f101eb740 10 client.5271 fetch_fsmap finished waiting for FSMap version 18 2024-04-22T06:32:12.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.843+0000 7f0f101eb740 10 client.5271 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-22T06:32:12.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.843+0000 7f0f101eb740 20 client.5271 populate_metadata read hostname 'smithi044' 2024-04-22T06:32:12.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.843+0000 7f0f101eb740 1 --2- 172.21.15.44:0/3324279107 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x560d66fc9f10 0x560d66fd60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:12.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.843+0000 7f0f101eb740 4 client.5271 mds_command: new command op to 4308 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T06:32:12.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.843+0000 7f0f101eb740 1 -- 172.21.15.44:0/3324279107 --> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] -- command(tid 0: {"prefix": "status"}) -- 0x560d66e793a0 con 0x560d66fc9f10 2024-04-22T06:32:12.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.843+0000 7f0f06ffd640 1 --2- 172.21.15.44:0/3324279107 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x560d66fc9f10 0x560d66fd60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:12.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.844+0000 7f0ee5ffb640 10 client.5271 ms_handle_connect on v2:172.21.15.70:6837/2915150040 2024-04-22T06:32:12.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.844+0000 7f0f06ffd640 1 --2- 172.21.15.44:0/3324279107 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x560d66fc9f10 0x560d66fd60a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4439c8cdca76807b server_cookie=49d806c363e9515b in_seq=0 out_seq=0 2024-04-22T06:32:12.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.844+0000 7f0ee5ffb640 1 -- 172.21.15.44:0/3324279107 <== mds.0 v2:172.21.15.70:6837/2915150040 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x560d66e793a0 con 0x560d66fc9f10 2024-04-22T06:32:12.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.844+0000 7f0ee5ffb640 10 client.5271 handle_command_reply: tid=0 2024-04-22T06:32:12.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.844+0000 7f0f101eb740 1 client.5271 shutdown 2024-04-22T06:32:12.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.844+0000 7f0f101eb740 1 -- 172.21.15.44:0/3324279107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d66f4de40 msgr2=0x560d66f29370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.844+0000 7f0f101eb740 1 --2- 172.21.15.44:0/3324279107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d66f4de40 0x560d66f29370 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0001140 tx=0x7f0ef0000ed0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.844+0000 7f0f101eb740 1 -- 172.21.15.44:0/3324279107 shutdown_connections 2024-04-22T06:32:12.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.844+0000 7f0f101eb740 1 --2- 172.21.15.44:0/3324279107 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560d66f541f0 0x560d66f298b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.844+0000 7f0f101eb740 1 --2- 172.21.15.44:0/3324279107 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x560d66fc9f10 0x560d66fd60a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.844+0000 7f0f101eb740 1 --2- 172.21.15.44:0/3324279107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d66f4de40 0x560d66f29370 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.844+0000 7f0f101eb740 1 --2- 172.21.15.44:0/3324279107 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d66f0eec0 0x560d66fc3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.844+0000 7f0ee5ffb640 0 client.5271 ms_handle_reset on v2:172.21.15.70:6837/2915150040 2024-04-22T06:32:12.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.844+0000 7f0f101eb740 1 -- 172.21.15.44:0/3324279107 >> 172.21.15.44:0/3324279107 conn(0x560d66f0aca0 msgr2=0x560d66f2a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:12.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.844+0000 7f0f101eb740 1 -- 172.21.15.44:0/3324279107 shutdown_connections 2024-04-22T06:32:12.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.844+0000 7f0f101eb740 1 -- 172.21.15.44:0/3324279107 wait complete. 2024-04-22T06:32:12.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.844+0000 7f0f101eb740 20 client.5271 trim_cache size 0 max 16384 2024-04-22T06:32:12.849 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-22T06:32:12.849 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "9b0a34ca-403d-4513-8a66-d3749ba559aa", 2024-04-22T06:32:12.849 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-04-22T06:32:12.849 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 4308, 2024-04-22T06:32:12.849 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-04-22T06:32:12.850 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-04-22T06:32:12.850 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-04-22T06:32:12.850 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 65.836790589000003, 2024-04-22T06:32:12.850 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 18, 2024-04-22T06:32:12.850 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 27, 2024-04-22T06:32:12.850 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 27, 2024-04-22T06:32:12.850 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 72.736713471000002 2024-04-22T06:32:12.850 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-22T06:32:12.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.845+0000 7f0f0d523640 1 -- 172.21.15.44:0/2097051416 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f0ee806c8f0 msgr2=0x7f0ee806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.845+0000 7f0f0d523640 1 --2- 172.21.15.44:0/2097051416 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f0ee806c8f0 0x7f0ee806edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0f000012b0 tx=0x7f0f00009040 comp rx=0 tx=0).stop 2024-04-22T06:32:12.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.845+0000 7f0f0d523640 1 -- 172.21.15.44:0/2097051416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f08155570 msgr2=0x7f0f080cd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:12.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.845+0000 7f0f0d523640 1 --2- 172.21.15.44:0/2097051416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f08155570 0x7f0f080cd880 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8009d90 tx=0x7f0ef8004a40 comp rx=0 tx=0).stop 2024-04-22T06:32:12.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.845+0000 7f0f0d523640 1 -- 172.21.15.44:0/2097051416 shutdown_connections 2024-04-22T06:32:12.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.845+0000 7f0f0d523640 1 --2- 172.21.15.44:0/2097051416 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f0ee806c8f0 0x7f0ee806edb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.845+0000 7f0f0d523640 1 --2- 172.21.15.44:0/2097051416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f08155570 0x7f0f080cd880 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.845+0000 7f0f0d523640 1 --2- 172.21.15.44:0/2097051416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f080cc8e0 0x7f0f080cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.845+0000 7f0f0d523640 1 --2- 172.21.15.44:0/2097051416 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f080cc4a0 0x7f0f080c6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:12.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.845+0000 7f0f0d523640 1 -- 172.21.15.44:0/2097051416 >> 172.21.15.44:0/2097051416 conn(0x7f0f08075870 msgr2=0x7f0f08154300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:12.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.845+0000 7f0f0d523640 1 -- 172.21.15.44:0/2097051416 shutdown_connections 2024-04-22T06:32:12.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:12.845+0000 7f0f0d523640 1 -- 172.21.15.44:0/2097051416 wait complete. 2024-04-22T06:32:12.854 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-04-22T06:32:27.855 INFO:tasks.quiescer.fs.[cephfs]:Total duration (52.2) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-22T06:32:27.855 INFO:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.1 2024-04-22T06:32:27.855 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T06:32:27.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.970+0000 7f82fb92b640 1 Processor -- start 2024-04-22T06:32:27.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.970+0000 7f82fb92b640 1 -- start start 2024-04-22T06:32:27.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.971+0000 7f82fb92b640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f82f41553c0 0x7f82f406b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:27.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.971+0000 7f82fb92b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f82f415f2c0 0x7f82f406be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:27.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.971+0000 7f82fb92b640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82f415b160 con 0x7f82f4070e70 2024-04-22T06:32:27.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.971+0000 7f82fb92b640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f82f415c050 con 0x7f82f415f2c0 2024-04-22T06:32:27.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.971+0000 7f82fb92b640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f82f41557a0 con 0x7f82f41553c0 2024-04-22T06:32:27.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.971+0000 7f82f8ec6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f82f415f2c0 0x7f82f406be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:27.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.971+0000 7f82f96c7640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f82f41553c0 0x7f82f406b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:27.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.971+0000 7f82f8ec6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f82f415f2c0 0x7f82f406be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:38834/0 (socket says 172.21.15.44:38834) 2024-04-22T06:32:27.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.971+0000 7f82f8ec6640 1 -- 172.21.15.44:0/2197380382 learned_addr learned my addr 172.21.15.44:0/2197380382 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:27.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.972+0000 7f82f96c7640 1 -- 172.21.15.44:0/2197380382 >> v2:172.21.15.70:3300/0 conn(0x7f82f415f2c0 msgr2=0x7f82f406be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:27.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.972+0000 7f82f96c7640 1 --2- 172.21.15.44:0/2197380382 >> v2:172.21.15.70:3300/0 conn(0x7f82f415f2c0 0x7f82f406be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:27.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.972+0000 7f82f96c7640 1 -- 172.21.15.44:0/2197380382 >> v1:172.21.15.70:6789/0 conn(0x7f82f4070e70 legacy=0x7f82f4071300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:27.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.972+0000 7f82f96c7640 1 -- 172.21.15.44:0/2197380382 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f415b440 con 0x7f82f41553c0 2024-04-22T06:32:27.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.972+0000 7f82ebfff640 1 -- 172.21.15.44:0/2197380382 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3256058820 0 0) 0x7f82f415b160 con 0x7f82f4070e70 2024-04-22T06:32:27.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.972+0000 7f82f8ec6640 1 --2- 172.21.15.44:0/2197380382 >> v2:172.21.15.70:3300/0 conn(0x7f82f415f2c0 0x7f82f406be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:32:27.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.973+0000 7f82f96c7640 1 --2- 172.21.15.44:0/2197380382 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f82f41553c0 0x7f82f406b930 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f82f00097a0 tx=0x7f82f002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=369238fd996eed0d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:27.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.973+0000 7f82ebfff640 1 -- 172.21.15.44:0/2197380382 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f0035020 con 0x7f82f41553c0 2024-04-22T06:32:27.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.973+0000 7f82ebfff640 1 -- 172.21.15.44:0/2197380382 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82f0031830 con 0x7f82f41553c0 2024-04-22T06:32:27.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.973+0000 7f82ebfff640 1 -- 172.21.15.44:0/2197380382 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f0031c00 con 0x7f82f41553c0 2024-04-22T06:32:27.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.973+0000 7f82fb92b640 1 -- 172.21.15.44:0/2197380382 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f82f41553c0 msgr2=0x7f82f406b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:27.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.973+0000 7f82fb92b640 1 --2- 172.21.15.44:0/2197380382 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f82f41553c0 0x7f82f406b930 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f82f00097a0 tx=0x7f82f002b8d0 comp rx=0 tx=0).stop 2024-04-22T06:32:27.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.973+0000 7f82fb92b640 1 -- 172.21.15.44:0/2197380382 shutdown_connections 2024-04-22T06:32:27.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.973+0000 7f82fb92b640 1 --2- 172.21.15.44:0/2197380382 >> v2:172.21.15.70:3300/0 conn(0x7f82f415f2c0 0x7f82f406be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:27.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.973+0000 7f82fb92b640 1 --2- 172.21.15.44:0/2197380382 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f82f41553c0 0x7f82f406b930 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:27.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.973+0000 7f82fb92b640 1 -- 172.21.15.44:0/2197380382 >> 172.21.15.44:0/2197380382 conn(0x7f82f4074cb0 msgr2=0x7f82f40750c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:27.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.974+0000 7f82fb92b640 1 -- 172.21.15.44:0/2197380382 shutdown_connections 2024-04-22T06:32:27.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.974+0000 7f82fb92b640 1 -- 172.21.15.44:0/2197380382 wait complete. 2024-04-22T06:32:27.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.975+0000 7f82fb92b640 1 Processor -- start 2024-04-22T06:32:27.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.975+0000 7f82fb92b640 1 -- start start 2024-04-22T06:32:27.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.975+0000 7f82fb92b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82f4070e70 0x7f82f415a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:27.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.975+0000 7f82fb92b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f41553c0 0x7f82f41f4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:27.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.975+0000 7f82fb92b640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f82f415f2c0 0x7f82f41f4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:27.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.975+0000 7f82fb92b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f82f415bce0 con 0x7f82f4070e70 2024-04-22T06:32:27.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.976+0000 7f82fb92b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f82f415ba00 con 0x7f82f41553c0 2024-04-22T06:32:27.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.976+0000 7f82fb92b640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f82f415c820 con 0x7f82f415f2c0 2024-04-22T06:32:27.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.976+0000 7f82f96c7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82f4070e70 0x7f82f415a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:27.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.976+0000 7f82f96c7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82f4070e70 0x7f82f415a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34716/0 (socket says 172.21.15.44:34716) 2024-04-22T06:32:27.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.976+0000 7f82f96c7640 1 -- 172.21.15.44:0/2035128767 learned_addr learned my addr 172.21.15.44:0/2035128767 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:27.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.976+0000 7f82f9ec8640 1 --2- 172.21.15.44:0/2035128767 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f82f415f2c0 0x7f82f41f4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:27.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.976+0000 7f82f8ec6640 1 --2- 172.21.15.44:0/2035128767 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f41553c0 0x7f82f41f4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:27.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.976+0000 7f82f96c7640 1 -- 172.21.15.44:0/2035128767 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f82f415f2c0 msgr2=0x7f82f41f4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:27.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.976+0000 7f82f96c7640 1 --2- 172.21.15.44:0/2035128767 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f82f415f2c0 0x7f82f41f4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:27.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.976+0000 7f82f96c7640 1 -- 172.21.15.44:0/2035128767 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f41553c0 msgr2=0x7f82f41f4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:27.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.976+0000 7f82f96c7640 1 --2- 172.21.15.44:0/2035128767 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f41553c0 0x7f82f41f4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:27.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.976+0000 7f82f96c7640 1 -- 172.21.15.44:0/2035128767 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f40cc880 con 0x7f82f4070e70 2024-04-22T06:32:27.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.976+0000 7f82f9ec8640 1 --2- 172.21.15.44:0/2035128767 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f82f415f2c0 0x7f82f41f4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:32:27.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.976+0000 7f82f96c7640 1 --2- 172.21.15.44:0/2035128767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82f4070e70 0x7f82f415a2b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f82f0009890 tx=0x7f82f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:27.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.976+0000 7f82e9ffb640 1 -- 172.21.15.44:0/2035128767 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f0035040 con 0x7f82f4070e70 2024-04-22T06:32:27.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.976+0000 7f82e9ffb640 1 -- 172.21.15.44:0/2035128767 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82f003b920 con 0x7f82f4070e70 2024-04-22T06:32:27.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.976+0000 7f82e9ffb640 1 -- 172.21.15.44:0/2035128767 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f003bd10 con 0x7f82f4070e70 2024-04-22T06:32:27.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.976+0000 7f82fb92b640 1 -- 172.21.15.44:0/2035128767 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82f415d2a0 con 0x7f82f4070e70 2024-04-22T06:32:27.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.977+0000 7f82fb92b640 1 -- 172.21.15.44:0/2035128767 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82f415d860 con 0x7f82f4070e70 2024-04-22T06:32:27.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.978+0000 7f82fb92b640 1 -- 172.21.15.44:0/2035128767 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82f4068cc0 con 0x7f82f4070e70 2024-04-22T06:32:27.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.978+0000 7f82e9ffb640 1 -- 172.21.15.44:0/2035128767 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f82f0038020 con 0x7f82f4070e70 2024-04-22T06:32:27.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.979+0000 7f82e9ffb640 1 --2- 172.21.15.44:0/2035128767 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f82d806c920 0x7f82d806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:27.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.979+0000 7f82e9ffb640 1 -- 172.21.15.44:0/2035128767 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f82f00ae090 con 0x7f82f4070e70 2024-04-22T06:32:27.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.979+0000 7f82f8ec6640 1 --2- 172.21.15.44:0/2035128767 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f82d806c920 0x7f82d806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:27.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.979+0000 7f82f8ec6640 1 --2- 172.21.15.44:0/2035128767 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f82d806c920 0x7f82d806ede0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f82f40cf3d0 tx=0x7f82e4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:27.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:27.983+0000 7f82e9ffb640 1 -- 172.21.15.44:0/2035128767 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f82f0031800 con 0x7f82f4070e70 2024-04-22T06:32:28.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.207+0000 7f82fb92b640 1 -- 172.21.15.44:0/2035128767 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82f40cf670 con 0x7f82f4070e70 2024-04-22T06:32:28.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.208+0000 7f82e9ffb640 1 -- 172.21.15.44:0/2035128767 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+6905 (secure 0 0 0) 0x7f82f007b9e0 con 0x7f82f4070e70 2024-04-22T06:32:28.208 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:32:28.208 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/1779271860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":1779271860},{"type":"v1","addr":"172.21.15.70:6836","nonce":1779271860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T06:31:06.356312+0000","modified":"2024-04-22T06:31:56.168744+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4308,"mds_1":4305,"mds_2":4293,"mds_3":4290,"mds_4":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":4,"incarnation":17,"state":"up:active","state_seq":15,"addr":"172.21.15.44:6834/1497302315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1497302315},{"type":"v1","addr":"172.21.15.44:6834","nonce":1497302315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4290":{"gid":4290,"name":"a","rank":3,"incarnation":15,"state":"up:active","state_seq":14,"addr":"172.21.15.44:6835/2383621014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2383621014},{"type":"v1","addr":"172.21.15.44:6835","nonce":2383621014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"e","rank":2,"incarnation":13,"state":"up:active","state_seq":12,"addr":"172.21.15.44:6837/915960587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":915960587},{"type":"v1","addr":"172.21.15.44:6837","nonce":915960587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4305":{"gid":4305,"name":"b","rank":1,"incarnation":9,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6838/924955785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":924955785},{"type":"v1","addr":"172.21.15.70:6838","nonce":924955785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4308":{"gid":4308,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.70:6839/2915150040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6837","nonce":2915150040},{"type":"v1","addr":"172.21.15.70:6839","nonce":2915150040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4308,"qdb_cluster":[4287,4290,4293,4305,4308]},"id":1}]} 2024-04-22T06:32:28.209 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 18 2024-04-22T06:32:28.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.210+0000 7f82fb92b640 1 -- 172.21.15.44:0/2035128767 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f82d806c920 msgr2=0x7f82d806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:28.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.210+0000 7f82fb92b640 1 --2- 172.21.15.44:0/2035128767 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f82d806c920 0x7f82d806ede0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f82f40cf3d0 tx=0x7f82e4008040 comp rx=0 tx=0).stop 2024-04-22T06:32:28.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.210+0000 7f82fb92b640 1 -- 172.21.15.44:0/2035128767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82f4070e70 msgr2=0x7f82f415a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:28.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.210+0000 7f82fb92b640 1 --2- 172.21.15.44:0/2035128767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82f4070e70 0x7f82f415a2b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f82f0009890 tx=0x7f82f00046c0 comp rx=0 tx=0).stop 2024-04-22T06:32:28.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.210+0000 7f82fb92b640 1 -- 172.21.15.44:0/2035128767 shutdown_connections 2024-04-22T06:32:28.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.210+0000 7f82fb92b640 1 --2- 172.21.15.44:0/2035128767 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f82d806c920 0x7f82d806ede0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:28.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.210+0000 7f82fb92b640 1 --2- 172.21.15.44:0/2035128767 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f82f415f2c0 0x7f82f41f4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:28.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.210+0000 7f82fb92b640 1 --2- 172.21.15.44:0/2035128767 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f41553c0 0x7f82f41f4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:28.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.210+0000 7f82fb92b640 1 --2- 172.21.15.44:0/2035128767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82f4070e70 0x7f82f415a2b0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:28.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.210+0000 7f82fb92b640 1 -- 172.21.15.44:0/2035128767 >> 172.21.15.44:0/2035128767 conn(0x7f82f4074cb0 msgr2=0x7f82f4075090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:28.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.210+0000 7f82fb92b640 1 -- 172.21.15.44:0/2035128767 shutdown_connections 2024-04-22T06:32:28.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.210+0000 7f82fb92b640 1 -- 172.21.15.44:0/2035128767 wait complete. 2024-04-22T06:32:28.219 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.4308 quiesce db / --timeout 60 --expiration 86.1 --await' 2024-04-22T06:32:28.220 DEBUG:teuthology.orchestra.run.smithi044:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4308 quiesce db / --timeout 60 --expiration 86.1 --await 2024-04-22T06:32:28.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.330+0000 7f00fe89a640 1 Processor -- start 2024-04-22T06:32:28.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.330+0000 7f00fe89a640 1 -- start start 2024-04-22T06:32:28.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.331+0000 7f00fe89a640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f00f815ef40 0x7f00f80cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:28.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.331+0000 7f00fe89a640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00f815bec0 con 0x7f00f80cb510 2024-04-22T06:32:28.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.331+0000 7f00fe89a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00f80cce70 con 0x7f00f8155010 2024-04-22T06:32:28.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.331+0000 7f00fe89a640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f00f81553f0 con 0x7f00f815ef40 2024-04-22T06:32:28.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.331+0000 7f00fce37640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f00f815ef40 0x7f00f80cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:28.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.331+0000 7f00f77fe640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f00f80cb510 0x7f00f815e830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:39226/0 (socket says 172.21.15.44:39226) 2024-04-22T06:32:28.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.331+0000 7f00f77fe640 1 -- 172.21.15.44:0/2431628516 learned_addr learned my addr 172.21.15.44:0/2431628516 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:32:28.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.331+0000 7f00fce37640 1 -- 172.21.15.44:0/2431628516 >> v1:172.21.15.70:6789/0 conn(0x7f00f8155010 legacy=0x7f00f80cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:28.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.331+0000 7f00fce37640 1 -- 172.21.15.44:0/2431628516 >> v1:172.21.15.44:6789/0 conn(0x7f00f80cb510 legacy=0x7f00f815e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:28.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.331+0000 7f00f6ffd640 1 -- 172.21.15.44:0/2431628516 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2004048173 0 0) 0x7f00f815bec0 con 0x7f00f80cb510 2024-04-22T06:32:28.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.331+0000 7f00fce37640 1 -- 172.21.15.44:0/2431628516 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f815b2b0 con 0x7f00f815ef40 2024-04-22T06:32:28.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.332+0000 7f00fce37640 1 --2- 172.21.15.44:0/2431628516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00f815ef40 0x7f00f80cc720 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f00f0009d90 tx=0x7f00f002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a1f73c2055d852 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:28.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.332+0000 7f00f6ffd640 1 -- 172.21.15.44:0/2431628516 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f0039070 con 0x7f00f815ef40 2024-04-22T06:32:28.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.332+0000 7f00f6ffd640 1 -- 172.21.15.44:0/2431628516 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00f002cd00 con 0x7f00f815ef40 2024-04-22T06:32:28.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.332+0000 7f00f6ffd640 1 -- 172.21.15.44:0/2431628516 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f0034660 con 0x7f00f815ef40 2024-04-22T06:32:28.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.333+0000 7f00fe89a640 1 -- 172.21.15.44:0/2431628516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00f815ef40 msgr2=0x7f00f80cc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:28.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.333+0000 7f00fe89a640 1 --2- 172.21.15.44:0/2431628516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00f815ef40 0x7f00f80cc720 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f00f0009d90 tx=0x7f00f002bf50 comp rx=0 tx=0).stop 2024-04-22T06:32:28.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.333+0000 7f00fe89a640 1 -- 172.21.15.44:0/2431628516 shutdown_connections 2024-04-22T06:32:28.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.333+0000 7f00fe89a640 1 --2- 172.21.15.44:0/2431628516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00f815ef40 0x7f00f80cc720 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:28.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.333+0000 7f00fe89a640 1 -- 172.21.15.44:0/2431628516 >> 172.21.15.44:0/2431628516 conn(0x7f00f806c770 msgr2=0x7f00f8074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:28.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.333+0000 7f00fe89a640 1 -- 172.21.15.44:0/2431628516 shutdown_connections 2024-04-22T06:32:28.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.333+0000 7f00fe89a640 1 -- 172.21.15.44:0/2431628516 wait complete. 2024-04-22T06:32:28.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.334+0000 7f00fe89a640 1 Processor -- start 2024-04-22T06:32:28.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.334+0000 7f00fe89a640 1 -- start start 2024-04-22T06:32:28.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.335+0000 7f00fe89a640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00f80cb510 0x7f00f814d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:28.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.335+0000 7f00fe89a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00f8155010 0x7f00f815c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:28.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.335+0000 7f00fe89a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00f815ef40 0x7f00f81f0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:28.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.335+0000 7f00fe89a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f00f815bb50 con 0x7f00f815ef40 2024-04-22T06:32:28.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.335+0000 7f00fe89a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f00f815b870 con 0x7f00f8155010 2024-04-22T06:32:28.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.335+0000 7f00fe89a640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f00f815af60 con 0x7f00f80cb510 2024-04-22T06:32:28.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.335+0000 7f00fce37640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00f815ef40 0x7f00f81f0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:28.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.335+0000 7f00fce37640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00f815ef40 0x7f00f81f0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34732/0 (socket says 172.21.15.44:34732) 2024-04-22T06:32:28.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.335+0000 7f00fce37640 1 -- 172.21.15.44:0/431913410 learned_addr learned my addr 172.21.15.44:0/431913410 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:28.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.335+0000 7f00f77fe640 1 --2- 172.21.15.44:0/431913410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00f8155010 0x7f00f815c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:28.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.335+0000 7f00f7fff640 1 --2- 172.21.15.44:0/431913410 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00f80cb510 0x7f00f814d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:28.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.335+0000 7f00fce37640 1 -- 172.21.15.44:0/431913410 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00f80cb510 msgr2=0x7f00f814d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:28.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.335+0000 7f00fce37640 1 --2- 172.21.15.44:0/431913410 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00f80cb510 0x7f00f814d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:28.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.335+0000 7f00fce37640 1 -- 172.21.15.44:0/431913410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00f8155010 msgr2=0x7f00f815c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:28.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.335+0000 7f00fce37640 1 --2- 172.21.15.44:0/431913410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00f8155010 0x7f00f815c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:28.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.335+0000 7f00fce37640 1 -- 172.21.15.44:0/431913410 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f815dd30 con 0x7f00f815ef40 2024-04-22T06:32:28.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.335+0000 7f00f77fe640 1 --2- 172.21.15.44:0/431913410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00f8155010 0x7f00f815c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:32:28.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.335+0000 7f00f7fff640 1 --2- 172.21.15.44:0/431913410 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00f80cb510 0x7f00f814d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:32:28.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.335+0000 7f00fce37640 1 --2- 172.21.15.44:0/431913410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00f815ef40 0x7f00f81f0e00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f00f0037040 tx=0x7f00f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:28.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.336+0000 7f00f4ff9640 1 -- 172.21.15.44:0/431913410 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f0044020 con 0x7f00f815ef40 2024-04-22T06:32:28.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.336+0000 7f00f4ff9640 1 -- 172.21.15.44:0/431913410 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00f003a040 con 0x7f00f815ef40 2024-04-22T06:32:28.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.336+0000 7f00f4ff9640 1 -- 172.21.15.44:0/431913410 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f0039050 con 0x7f00f815ef40 2024-04-22T06:32:28.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.336+0000 7f00fe89a640 1 -- 172.21.15.44:0/431913410 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00f815dfd0 con 0x7f00f815ef40 2024-04-22T06:32:28.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.336+0000 7f00fe89a640 1 -- 172.21.15.44:0/431913410 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00f81592c0 con 0x7f00f815ef40 2024-04-22T06:32:28.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.337+0000 7f00f4ff9640 1 -- 172.21.15.44:0/431913410 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f00f0003bb0 con 0x7f00f815ef40 2024-04-22T06:32:28.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.337+0000 7f00f4ff9640 1 --2- 172.21.15.44:0/431913410 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f00e406c8f0 0x7f00e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:28.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.338+0000 7f00f4ff9640 1 -- 172.21.15.44:0/431913410 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f00f00ae620 con 0x7f00f815ef40 2024-04-22T06:32:28.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.338+0000 7f00f7fff640 1 --2- 172.21.15.44:0/431913410 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f00e406c8f0 0x7f00e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:28.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.338+0000 7f00f7fff640 1 --2- 172.21.15.44:0/431913410 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f00e406c8f0 0x7f00e406edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f00ec007fd0 tx=0x7f00ec009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:28.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.338+0000 7f0101569740 1 Processor -- start 2024-04-22T06:32:28.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.338+0000 7f0101569740 1 -- start start 2024-04-22T06:32:28.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.339+0000 7f0101569740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f27b8d2200 0x55f27b8d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:28.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.339+0000 7f0101569740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f27b91c080 0x55f27b8d2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:28.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.339+0000 7f0101569740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f27b8d0c60 0x55f27b911b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:28.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.339+0000 7f0101569740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f27b8330b0 con 0x55f27b8d2200 2024-04-22T06:32:28.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.339+0000 7f0101569740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55f27b7e4680 con 0x55f27b8d0c60 2024-04-22T06:32:28.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.339+0000 7f0101569740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55f27b7f0c60 con 0x55f27b91c080 2024-04-22T06:32:28.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.339+0000 7f00f7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f27b8d2200 0x55f27b8d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:28.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.339+0000 7f00f7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f27b8d2200 0x55f27b8d25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34734/0 (socket says 172.21.15.44:34734) 2024-04-22T06:32:28.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.339+0000 7f00f7fff640 1 -- 172.21.15.44:0/431008148 learned_addr learned my addr 172.21.15.44:0/431008148 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:28.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.339+0000 7f00f77fe640 1 --2- 172.21.15.44:0/431008148 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f27b91c080 0x55f27b8d2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:28.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.339+0000 7f00fce37640 1 --2- 172.21.15.44:0/431008148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f27b8d0c60 0x55f27b911b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:28.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.340+0000 7f00f7fff640 1 -- 172.21.15.44:0/431008148 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f27b91c080 msgr2=0x55f27b8d2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:28.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.340+0000 7f00f7fff640 1 --2- 172.21.15.44:0/431008148 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f27b91c080 0x55f27b8d2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:28.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.340+0000 7f00f7fff640 1 -- 172.21.15.44:0/431008148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f27b8d0c60 msgr2=0x55f27b911b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:28.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.340+0000 7f00f7fff640 1 --2- 172.21.15.44:0/431008148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f27b8d0c60 0x55f27b911b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:28.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.340+0000 7f00f7fff640 1 -- 172.21.15.44:0/431008148 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f27b615c90 con 0x55f27b8d2200 2024-04-22T06:32:28.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.340+0000 7f00f7fff640 1 --2- 172.21.15.44:0/431008148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f27b8d2200 0x55f27b8d25e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f00ec005360 tx=0x7f00ec010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88582f9389d89289 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:28.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.340+0000 7f00ddffb640 1 -- 172.21.15.44:0/431008148 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec003a00 con 0x55f27b8d2200 2024-04-22T06:32:28.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.340+0000 7f00ddffb640 1 -- 172.21.15.44:0/431008148 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00ec003ba0 con 0x55f27b8d2200 2024-04-22T06:32:28.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.340+0000 7f00ddffb640 1 -- 172.21.15.44:0/431008148 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec020450 con 0x55f27b8d2200 2024-04-22T06:32:28.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.340+0000 7f0101569740 1 -- 172.21.15.44:0/431008148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f27b8d2200 msgr2=0x55f27b8d25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:28.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.340+0000 7f0101569740 1 --2- 172.21.15.44:0/431008148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f27b8d2200 0x55f27b8d25e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f00ec005360 tx=0x7f00ec010000 comp rx=0 tx=0).stop 2024-04-22T06:32:28.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.340+0000 7f0101569740 1 -- 172.21.15.44:0/431008148 shutdown_connections 2024-04-22T06:32:28.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.340+0000 7f0101569740 1 --2- 172.21.15.44:0/431008148 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f27b91c080 0x55f27b8d2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:28.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.340+0000 7f0101569740 1 --2- 172.21.15.44:0/431008148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f27b8d0c60 0x55f27b911b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:28.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.340+0000 7f0101569740 1 --2- 172.21.15.44:0/431008148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f27b8d2200 0x55f27b8d25e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:28.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.341+0000 7f0101569740 1 -- 172.21.15.44:0/431008148 >> 172.21.15.44:0/431008148 conn(0x55f27b8cec00 msgr2=0x55f27b8d12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:28.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.341+0000 7f0101569740 1 -- 172.21.15.44:0/431008148 shutdown_connections 2024-04-22T06:32:28.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.341+0000 7f0101569740 1 -- 172.21.15.44:0/431008148 wait complete. 2024-04-22T06:32:28.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.341+0000 7f0101569740 1 -- start start 2024-04-22T06:32:28.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.341+0000 7f0101569740 1 Processor -- start 2024-04-22T06:32:28.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.342+0000 7f0101569740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f27b8d2200 0x55f27b926850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:28.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.342+0000 7f0101569740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f27b91c080 0x55f27b91d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:28.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.342+0000 7f0101569740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f27b910320 0x55f27b8f6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:28.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.342+0000 7f0101569740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f27b836400 con 0x55f27b910320 2024-04-22T06:32:28.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.342+0000 7f0101569740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55f27b7f2840 con 0x55f27b8d2200 2024-04-22T06:32:28.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.342+0000 7f0101569740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55f27b83b720 con 0x55f27b91c080 2024-04-22T06:32:28.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.342+0000 7f0101569740 10 client.0 _ll_register_callbacks cb 0x55f27b843ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T06:32:28.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.342+0000 7f00fce37640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f27b910320 0x55f27b8f6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:28.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.342+0000 7f00fce37640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f27b910320 0x55f27b8f6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34738/0 (socket says 172.21.15.44:34738) 2024-04-22T06:32:28.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.342+0000 7f00fce37640 1 -- 172.21.15.44:0/3731846981 learned_addr learned my addr 172.21.15.44:0/3731846981 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:28.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.342+0000 7f00f7fff640 1 --2- 172.21.15.44:0/3731846981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f27b8d2200 0x55f27b926850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:28.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.342+0000 7f00f77fe640 1 --2- 172.21.15.44:0/3731846981 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f27b91c080 0x55f27b91d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:28.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.342+0000 7f00fce37640 1 -- 172.21.15.44:0/3731846981 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f27b91c080 msgr2=0x55f27b91d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:28.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.342+0000 7f00fce37640 1 --2- 172.21.15.44:0/3731846981 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f27b91c080 0x55f27b91d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:28.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.342+0000 7f00fce37640 1 -- 172.21.15.44:0/3731846981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f27b8d2200 msgr2=0x55f27b926850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:28.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.342+0000 7f00fce37640 1 --2- 172.21.15.44:0/3731846981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f27b8d2200 0x55f27b926850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:28.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.342+0000 7f00fce37640 1 -- 172.21.15.44:0/3731846981 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f27b676420 con 0x55f27b910320 2024-04-22T06:32:28.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.343+0000 7f00fce37640 1 --2- 172.21.15.44:0/3731846981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f27b910320 0x55f27b8f6040 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f00f002c560 tx=0x7f00f0080450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=996899425eabbda0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:28.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.343+0000 7f00d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T06:32:28.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.343+0000 7f00d6ffd640 1 -- 172.21.15.44:0/3731846981 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f00ac430 con 0x55f27b910320 2024-04-22T06:32:28.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.343+0000 7f00d6ffd640 1 -- 172.21.15.44:0/3731846981 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f00f00acb70 con 0x55f27b910320 2024-04-22T06:32:28.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.343+0000 7f0101569740 1 -- 172.21.15.44:0/3731846981 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00ec003ba0 con 0x55f27b910320 2024-04-22T06:32:28.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.343+0000 7f00d6ffd640 1 -- 172.21.15.44:0/3731846981 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f00f00acd70 con 0x55f27b910320 2024-04-22T06:32:28.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.343+0000 7f0101569740 10 client.5289 fetch_fsmap learned FSMap version 18 2024-04-22T06:32:28.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.343+0000 7f0101569740 1 -- 172.21.15.44:0/3731846981 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55f27b509580 con 0x55f27b910320 2024-04-22T06:32:28.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.343+0000 7f00d6ffd640 1 -- 172.21.15.44:0/3731846981 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00ec003ba0 con 0x55f27b910320 2024-04-22T06:32:28.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.343+0000 7f00d6ffd640 1 -- 172.21.15.44:0/3731846981 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 18) ==== 4792+0+0 (secure 0 0 0) 0x7f00f009cc20 con 0x55f27b910320 2024-04-22T06:32:28.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.343+0000 7f0101569740 10 client.5289 fetch_fsmap finished waiting for FSMap version 18 2024-04-22T06:32:28.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.343+0000 7f0101569740 10 client.5289 resolve_mds: validated gid 4308 aka daemon mds.d 2024-04-22T06:32:28.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.343+0000 7f0101569740 20 client.5289 populate_metadata read hostname 'smithi044' 2024-04-22T06:32:28.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.343+0000 7f0101569740 1 --2- 172.21.15.44:0/3731846981 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x55f27b8fba30 0x55f27b922ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:28.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.343+0000 7f0101569740 4 client.5289 mds_command: new command op to 4308 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T06:32:28.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.343+0000 7f0101569740 1 -- 172.21.15.44:0/3731846981 --> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f27b844b10 con 0x55f27b8fba30 2024-04-22T06:32:28.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.343+0000 7f00f7fff640 1 --2- 172.21.15.44:0/3731846981 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x55f27b8fba30 0x55f27b922ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:28.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.344+0000 7f00f7fff640 1 --2- 172.21.15.44:0/3731846981 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x55f27b8fba30 0x55f27b922ee0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cd05b8df3148041 server_cookie=213f21aaea7aa02f in_seq=0 out_seq=0 2024-04-22T06:32:28.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.344+0000 7f00d6ffd640 10 client.5289 ms_handle_connect on v2:172.21.15.70:6837/2915150040 2024-04-22T06:32:28.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.344+0000 7f00d6ffd640 1 -- 172.21.15.44:0/3731846981 <== mds.0 v2:172.21.15.70:6837/2915150040 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f27b844b10 con 0x55f27b8fba30 2024-04-22T06:32:28.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.344+0000 7f00d6ffd640 10 client.5289 handle_command_reply: tid=0 2024-04-22T06:32:28.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.345+0000 7f0101569740 1 client.5289 shutdown 2024-04-22T06:32:28.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.345+0000 7f0101569740 1 -- 172.21.15.44:0/3731846981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f27b910320 msgr2=0x55f27b8f6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:28.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.345+0000 7f0101569740 1 --2- 172.21.15.44:0/3731846981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f27b910320 0x55f27b8f6040 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f00f002c560 tx=0x7f00f0080450 comp rx=0 tx=0).stop 2024-04-22T06:32:28.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.345+0000 7f0101569740 1 -- 172.21.15.44:0/3731846981 shutdown_connections 2024-04-22T06:32:28.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.345+0000 7f0101569740 1 --2- 172.21.15.44:0/3731846981 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f27b91c080 0x55f27b91d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:28.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.345+0000 7f0101569740 1 --2- 172.21.15.44:0/3731846981 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x55f27b8fba30 0x55f27b922ee0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:28.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.345+0000 7f0101569740 1 --2- 172.21.15.44:0/3731846981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f27b910320 0x55f27b8f6040 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:28.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.345+0000 7f0101569740 1 --2- 172.21.15.44:0/3731846981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f27b8d2200 0x55f27b926850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:28.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.345+0000 7f00d6ffd640 0 client.5289 ms_handle_reset on v2:172.21.15.70:6837/2915150040 2024-04-22T06:32:28.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.345+0000 7f0101569740 1 -- 172.21.15.44:0/3731846981 >> 172.21.15.44:0/3731846981 conn(0x55f27b8d0c60 msgr2=0x55f27b8cb940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:28.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.345+0000 7f0101569740 1 -- 172.21.15.44:0/3731846981 shutdown_connections 2024-04-22T06:32:28.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.345+0000 7f0101569740 1 -- 172.21.15.44:0/3731846981 wait complete. 2024-04-22T06:32:28.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.345+0000 7f0101569740 20 client.5289 trim_cache size 0 max 16384 2024-04-22T06:32:28.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.362+0000 7f0101569740 1 Processor -- start 2024-04-22T06:32:28.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.362+0000 7f0101569740 1 -- start start 2024-04-22T06:32:28.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.363+0000 7f0101569740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f27b8d2200 0x55f27b8d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:28.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.363+0000 7f0101569740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f27b910320 0x55f27b8cf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:28.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.363+0000 7f0101569740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f27b91c080 0x55f27b8cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:28.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.363+0000 7f0101569740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f00f81ffad0 con 0x55f27b91c080 2024-04-22T06:32:28.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.363+0000 7f0101569740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f00f81fdf90 con 0x55f27b910320 2024-04-22T06:32:28.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.363+0000 7f0101569740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f00f81ff400 con 0x55f27b8d2200 2024-04-22T06:32:28.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.363+0000 7f00fce37640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f27b91c080 0x55f27b8cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:28.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.363+0000 7f00fce37640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f27b91c080 0x55f27b8cfc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34744/0 (socket says 172.21.15.44:34744) 2024-04-22T06:32:28.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.363+0000 7f00fce37640 1 -- 172.21.15.44:0/537249001 learned_addr learned my addr 172.21.15.44:0/537249001 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:28.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.364+0000 7f00f77fe640 1 --2- 172.21.15.44:0/537249001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f27b910320 0x55f27b8cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:28.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.364+0000 7f00fce37640 1 -- 172.21.15.44:0/537249001 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f27b8d2200 msgr2=0x55f27b8d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:28.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.364+0000 7f00fce37640 1 --2- 172.21.15.44:0/537249001 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f27b8d2200 0x55f27b8d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:28.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.364+0000 7f00fce37640 1 -- 172.21.15.44:0/537249001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f27b910320 msgr2=0x55f27b8cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:28.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.364+0000 7f00fce37640 1 --2- 172.21.15.44:0/537249001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f27b910320 0x55f27b8cf700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:28.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.364+0000 7f00fce37640 1 -- 172.21.15.44:0/537249001 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f27b8d3da0 con 0x55f27b91c080 2024-04-22T06:32:28.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.364+0000 7f00f77fe640 1 --2- 172.21.15.44:0/537249001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f27b910320 0x55f27b8cf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:32:28.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.364+0000 7f00fce37640 1 --2- 172.21.15.44:0/537249001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f27b91c080 0x55f27b8cfc40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f00f007bce0 tx=0x7f00f00ae050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd78734e07d5a1cc server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:28.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.364+0000 7f00dd7fa640 1 -- 172.21.15.44:0/537249001 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f009f070 con 0x55f27b91c080 2024-04-22T06:32:28.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.364+0000 7f00dd7fa640 1 -- 172.21.15.44:0/537249001 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00f00a1070 con 0x55f27b91c080 2024-04-22T06:32:28.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.364+0000 7f00dd7fa640 1 -- 172.21.15.44:0/537249001 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f007c6e0 con 0x55f27b91c080 2024-04-22T06:32:28.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.364+0000 7f0101569740 1 -- 172.21.15.44:0/537249001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f27b91c080 msgr2=0x55f27b8cfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:28.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.364+0000 7f0101569740 1 --2- 172.21.15.44:0/537249001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f27b91c080 0x55f27b8cfc40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f00f007bce0 tx=0x7f00f00ae050 comp rx=0 tx=0).stop 2024-04-22T06:32:28.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.364+0000 7f0101569740 1 -- 172.21.15.44:0/537249001 shutdown_connections 2024-04-22T06:32:28.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.364+0000 7f0101569740 1 --2- 172.21.15.44:0/537249001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f27b91c080 0x55f27b8cfc40 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:28.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.364+0000 7f0101569740 1 --2- 172.21.15.44:0/537249001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f27b910320 0x55f27b8cf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:28.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.364+0000 7f0101569740 1 --2- 172.21.15.44:0/537249001 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f27b8d2200 0x55f27b8d2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:28.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.364+0000 7f0101569740 1 -- 172.21.15.44:0/537249001 >> 172.21.15.44:0/537249001 conn(0x55f27b8d0c60 msgr2=0x55f27b919d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:28.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.364+0000 7f0101569740 1 -- 172.21.15.44:0/537249001 shutdown_connections 2024-04-22T06:32:28.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.365+0000 7f0101569740 1 -- 172.21.15.44:0/537249001 wait complete. 2024-04-22T06:32:28.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.365+0000 7f0101569740 1 -- start start 2024-04-22T06:32:28.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.365+0000 7f0101569740 1 Processor -- start 2024-04-22T06:32:28.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.366+0000 7f0101569740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f27b8d2200 0x55f27b98cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:28.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.366+0000 7f0101569740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f27b910320 0x55f27b920270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:28.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.366+0000 7f0101569740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f27b91c080 0x55f27b986930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:28.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.366+0000 7f0101569740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f00f81fed30 con 0x55f27b91c080 2024-04-22T06:32:28.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.366+0000 7f0101569740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f00f81fe660 con 0x55f27b910320 2024-04-22T06:32:28.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.366+0000 7f0101569740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f00f8147650 con 0x55f27b8d2200 2024-04-22T06:32:28.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.366+0000 7f0101569740 10 client.0 _ll_register_callbacks cb 0x7f00f0009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T06:32:28.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.366+0000 7f00fce37640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f27b91c080 0x55f27b986930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:28.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.366+0000 7f00fce37640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f27b91c080 0x55f27b986930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34760/0 (socket says 172.21.15.44:34760) 2024-04-22T06:32:28.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.366+0000 7f00fce37640 1 -- 172.21.15.44:0/3009443973 learned_addr learned my addr 172.21.15.44:0/3009443973 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:28.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.366+0000 7f00f7fff640 1 --2- 172.21.15.44:0/3009443973 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f27b8d2200 0x55f27b98cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:28.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.366+0000 7f00f77fe640 1 --2- 172.21.15.44:0/3009443973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f27b910320 0x55f27b920270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:28.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.366+0000 7f00fce37640 1 -- 172.21.15.44:0/3009443973 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f27b8d2200 msgr2=0x55f27b98cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:28.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.366+0000 7f00fce37640 1 --2- 172.21.15.44:0/3009443973 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f27b8d2200 0x55f27b98cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:28.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.366+0000 7f00fce37640 1 -- 172.21.15.44:0/3009443973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f27b910320 msgr2=0x55f27b920270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:28.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.366+0000 7f00fce37640 1 --2- 172.21.15.44:0/3009443973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f27b910320 0x55f27b920270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:28.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.366+0000 7f00fce37640 1 -- 172.21.15.44:0/3009443973 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f27b91b500 con 0x55f27b91c080 2024-04-22T06:32:28.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.367+0000 7f00fce37640 1 --2- 172.21.15.44:0/3009443973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f27b91c080 0x55f27b986930 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f00f002c560 tx=0x7f00f00ac6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f04bee57d0edb60f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:28.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.367+0000 7f00d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T06:32:28.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.367+0000 7f00d6ffd640 1 -- 172.21.15.44:0/3009443973 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f0089020 con 0x55f27b91c080 2024-04-22T06:32:28.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.367+0000 7f00d6ffd640 1 -- 172.21.15.44:0/3009443973 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f00f007a080 con 0x55f27b91c080 2024-04-22T06:32:28.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.367+0000 7f0101569740 1 -- 172.21.15.44:0/3009443973 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00f00a1070 con 0x55f27b91c080 2024-04-22T06:32:28.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.367+0000 7f00d6ffd640 1 -- 172.21.15.44:0/3009443973 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f00f009f070 con 0x55f27b91c080 2024-04-22T06:32:28.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.367+0000 7f0101569740 10 client.5295 fetch_fsmap learned FSMap version 18 2024-04-22T06:32:28.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.367+0000 7f0101569740 1 -- 172.21.15.44:0/3009443973 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55f27b91b500 con 0x55f27b91c080 2024-04-22T06:32:28.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.367+0000 7f00d6ffd640 1 -- 172.21.15.44:0/3009443973 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00f00a1070 con 0x55f27b91c080 2024-04-22T06:32:28.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.367+0000 7f00d6ffd640 1 -- 172.21.15.44:0/3009443973 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 18) ==== 4792+0+0 (secure 0 0 0) 0x7f00f007b430 con 0x55f27b91c080 2024-04-22T06:32:28.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.367+0000 7f0101569740 10 client.5295 fetch_fsmap finished waiting for FSMap version 18 2024-04-22T06:32:28.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.367+0000 7f0101569740 10 client.5295 resolve_mds: validated gid 4308 aka daemon mds.d 2024-04-22T06:32:28.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.367+0000 7f0101569740 20 client.5295 populate_metadata read hostname 'smithi044' 2024-04-22T06:32:28.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.367+0000 7f0101569740 1 --2- 172.21.15.44:0/3009443973 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x55f27b90c950 0x55f27b986e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:28.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.367+0000 7f0101569740 4 client.5295 mds_command: new command op to 4308 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 86.1, "await": true}] 2024-04-22T06:32:28.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.367+0000 7f0101569740 1 -- 172.21.15.44:0/3009443973 --> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 86.1, "await": true}) -- 0x55f27b83e980 con 0x55f27b90c950 2024-04-22T06:32:28.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.367+0000 7f00f7fff640 1 --2- 172.21.15.44:0/3009443973 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x55f27b90c950 0x55f27b986e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:28.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.368+0000 7f00d6ffd640 10 client.5295 ms_handle_connect on v2:172.21.15.70:6837/2915150040 2024-04-22T06:32:28.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.368+0000 7f00f7fff640 1 --2- 172.21.15.44:0/3009443973 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x55f27b90c950 0x55f27b986e70 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3264ff5dbfdfa22a server_cookie=94aaa54a4beb4b13 in_seq=0 out_seq=0 2024-04-22T06:32:28.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.370+0000 7f00d6ffd640 1 -- 172.21.15.44:0/3009443973 <== mds.0 v2:172.21.15.70:6837/2915150040 1 ==== command_reply(tid 0: 0 ) ==== 8+0+587 (crc 0 0 0) 0x55f27b83e980 con 0x55f27b90c950 2024-04-22T06:32:28.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.370+0000 7f00d6ffd640 10 client.5295 handle_command_reply: tid=0 2024-04-22T06:32:28.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.370+0000 7f0101569740 1 client.5295 shutdown 2024-04-22T06:32:28.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.371+0000 7f0101569740 1 -- 172.21.15.44:0/3009443973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f27b91c080 msgr2=0x55f27b986930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:28.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.371+0000 7f0101569740 1 --2- 172.21.15.44:0/3009443973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f27b91c080 0x55f27b986930 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f00f002c560 tx=0x7f00f00ac6b0 comp rx=0 tx=0).stop 2024-04-22T06:32:28.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.371+0000 7f0101569740 1 -- 172.21.15.44:0/3009443973 shutdown_connections 2024-04-22T06:32:28.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.371+0000 7f0101569740 1 --2- 172.21.15.44:0/3009443973 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x55f27b90c950 0x55f27b986e70 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:28.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.371+0000 7f0101569740 1 --2- 172.21.15.44:0/3009443973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f27b91c080 0x55f27b986930 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:28.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.371+0000 7f0101569740 1 --2- 172.21.15.44:0/3009443973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f27b910320 0x55f27b920270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:28.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.371+0000 7f0101569740 1 --2- 172.21.15.44:0/3009443973 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f27b8d2200 0x55f27b98cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:28.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.371+0000 7f0101569740 1 -- 172.21.15.44:0/3009443973 >> 172.21.15.44:0/3009443973 conn(0x55f27b8d0c60 msgr2=0x55f27b919d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:28.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.371+0000 7f00d6ffd640 0 client.5295 ms_handle_reset on v2:172.21.15.70:6837/2915150040 2024-04-22T06:32:28.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.371+0000 7f0101569740 1 -- 172.21.15.44:0/3009443973 shutdown_connections 2024-04-22T06:32:28.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.371+0000 7f0101569740 1 -- 172.21.15.44:0/3009443973 wait complete. 2024-04-22T06:32:28.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.371+0000 7f0101569740 20 client.5295 trim_cache size 0 max 16384 2024-04-22T06:32:28.374 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-22T06:32:28.374 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 18, 2024-04-22T06:32:28.374 INFO:teuthology.orchestra.run.smithi044.stdout: "leader": 4308, 2024-04-22T06:32:28.374 INFO:teuthology.orchestra.run.smithi044.stdout: "set_version": 2, 2024-04-22T06:32:28.374 INFO:teuthology.orchestra.run.smithi044.stdout: "sets": { 2024-04-22T06:32:28.374 INFO:teuthology.orchestra.run.smithi044.stdout: "ab138efc": { 2024-04-22T06:32:28.374 INFO:teuthology.orchestra.run.smithi044.stdout: "version": 2, 2024-04-22T06:32:28.374 INFO:teuthology.orchestra.run.smithi044.stdout: "age_ref": 0.0, 2024-04-22T06:32:28.374 INFO:teuthology.orchestra.run.smithi044.stdout: "state": { 2024-04-22T06:32:28.374 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "QUIESCED", 2024-04-22T06:32:28.374 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.0 2024-04-22T06:32:28.374 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-22T06:32:28.374 INFO:teuthology.orchestra.run.smithi044.stdout: "timeout": 60.0, 2024-04-22T06:32:28.375 INFO:teuthology.orchestra.run.smithi044.stdout: "expiration": 86.1, 2024-04-22T06:32:28.375 INFO:teuthology.orchestra.run.smithi044.stdout: "members": { 2024-04-22T06:32:28.375 INFO:teuthology.orchestra.run.smithi044.stdout: "file:/": { 2024-04-22T06:32:28.375 INFO:teuthology.orchestra.run.smithi044.stdout: "excluded": false, 2024-04-22T06:32:28.375 INFO:teuthology.orchestra.run.smithi044.stdout: "state": { 2024-04-22T06:32:28.375 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "QUIESCED", 2024-04-22T06:32:28.375 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.0 2024-04-22T06:32:28.375 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-22T06:32:28.375 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-22T06:32:28.375 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-22T06:32:28.375 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-22T06:32:28.375 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-22T06:32:28.375 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-22T06:32:28.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.371+0000 7f00fe89a640 1 -- 172.21.15.44:0/431913410 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f00e406c8f0 msgr2=0x7f00e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:28.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.371+0000 7f00fe89a640 1 --2- 172.21.15.44:0/431913410 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f00e406c8f0 0x7f00e406edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f00ec007fd0 tx=0x7f00ec009040 comp rx=0 tx=0).stop 2024-04-22T06:32:28.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.371+0000 7f00fe89a640 1 -- 172.21.15.44:0/431913410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00f815ef40 msgr2=0x7f00f81f0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:28.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.371+0000 7f00fe89a640 1 --2- 172.21.15.44:0/431913410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00f815ef40 0x7f00f81f0e00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f00f0037040 tx=0x7f00f0004a40 comp rx=0 tx=0).stop 2024-04-22T06:32:28.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.371+0000 7f00fe89a640 1 -- 172.21.15.44:0/431913410 shutdown_connections 2024-04-22T06:32:28.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.371+0000 7f00fe89a640 1 --2- 172.21.15.44:0/431913410 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f00e406c8f0 0x7f00e406edb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:28.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.371+0000 7f00fe89a640 1 --2- 172.21.15.44:0/431913410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00f815ef40 0x7f00f81f0e00 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:28.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.371+0000 7f00fe89a640 1 --2- 172.21.15.44:0/431913410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00f8155010 0x7f00f815c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:28.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.371+0000 7f00fe89a640 1 --2- 172.21.15.44:0/431913410 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00f80cb510 0x7f00f814d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:28.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.371+0000 7f00fe89a640 1 -- 172.21.15.44:0/431913410 >> 172.21.15.44:0/431913410 conn(0x7f00f806c770 msgr2=0x7f00f80736b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:28.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.372+0000 7f00fe89a640 1 -- 172.21.15.44:0/431913410 shutdown_connections 2024-04-22T06:32:28.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:28.372+0000 7f00fe89a640 1 -- 172.21.15.44:0/431913410 wait complete. 2024-04-22T06:32:28.381 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced, set_id: ab138efc, quiesce took 0.5 seconds. Will release after: 26.1 2024-04-22T06:32:38.688 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T06:32:38.691 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T06:32:54.482 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T06:32:54.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.597+0000 7f9717396640 1 Processor -- start 2024-04-22T06:32:54.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.598+0000 7f9717396640 1 -- start start 2024-04-22T06:32:54.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.598+0000 7f9717396640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9710155240 0x7f971015f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:54.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.598+0000 7f9717396640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f97100cb320 0x7f97100d0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:54.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.598+0000 7f9717396640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f971015b200 con 0x7f97100d1400 2024-04-22T06:32:54.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.598+0000 7f9717396640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f971015f610 con 0x7f97100cb320 2024-04-22T06:32:54.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.598+0000 7f9717396640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f971015c060 con 0x7f9710155240 2024-04-22T06:32:54.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.599+0000 7f9714931640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f97100cb320 0x7f97100d0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:54.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.599+0000 7f9714931640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f97100cb320 0x7f97100d0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60160/0 (socket says 172.21.15.44:60160) 2024-04-22T06:32:54.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.599+0000 7f9714931640 1 -- 172.21.15.44:0/4056292266 learned_addr learned my addr 172.21.15.44:0/4056292266 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:54.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.599+0000 7f9715132640 1 --2- 172.21.15.44:0/4056292266 >> v2:172.21.15.70:3300/0 conn(0x7f9710155240 0x7f971015f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:54.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.599+0000 7f9714931640 1 -- 172.21.15.44:0/4056292266 >> v2:172.21.15.70:3300/0 conn(0x7f9710155240 msgr2=0x7f971015f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:54.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.599+0000 7f9714931640 1 --2- 172.21.15.44:0/4056292266 >> v2:172.21.15.70:3300/0 conn(0x7f9710155240 0x7f971015f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:54.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.599+0000 7f9714931640 1 -- 172.21.15.44:0/4056292266 >> v1:172.21.15.70:6789/0 conn(0x7f97100d1400 legacy=0x7f97100cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:54.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.599+0000 7f9714931640 1 -- 172.21.15.44:0/4056292266 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f971015b4e0 con 0x7f97100cb320 2024-04-22T06:32:54.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.599+0000 7f9715132640 1 --2- 172.21.15.44:0/4056292266 >> v2:172.21.15.70:3300/0 conn(0x7f9710155240 0x7f971015f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:32:54.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.600+0000 7f9714931640 1 --2- 172.21.15.44:0/4056292266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97100cb320 0x7f97100d0ec0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9710067fc0 tx=0x7f970402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68bd4feea6a1da1f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:54.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.600+0000 7f970ffff640 1 -- 172.21.15.44:0/4056292266 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9704035020 con 0x7f97100cb320 2024-04-22T06:32:54.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.600+0000 7f970ffff640 1 -- 172.21.15.44:0/4056292266 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9704031870 con 0x7f97100cb320 2024-04-22T06:32:54.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.600+0000 7f970ffff640 1 -- 172.21.15.44:0/4056292266 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9704031c40 con 0x7f97100cb320 2024-04-22T06:32:54.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.600+0000 7f9717396640 1 -- 172.21.15.44:0/4056292266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97100cb320 msgr2=0x7f97100d0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:54.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.600+0000 7f9717396640 1 --2- 172.21.15.44:0/4056292266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97100cb320 0x7f97100d0ec0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9710067fc0 tx=0x7f970402b910 comp rx=0 tx=0).stop 2024-04-22T06:32:54.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.600+0000 7f9717396640 1 -- 172.21.15.44:0/4056292266 shutdown_connections 2024-04-22T06:32:54.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.600+0000 7f9717396640 1 --2- 172.21.15.44:0/4056292266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97100cb320 0x7f97100d0ec0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:54.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.601+0000 7f9717396640 1 --2- 172.21.15.44:0/4056292266 >> v2:172.21.15.70:3300/0 conn(0x7f9710155240 0x7f971015f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:54.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.601+0000 7f9717396640 1 -- 172.21.15.44:0/4056292266 >> 172.21.15.44:0/4056292266 conn(0x7f97100750d0 msgr2=0x7f97100754e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:54.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.601+0000 7f9717396640 1 -- 172.21.15.44:0/4056292266 shutdown_connections 2024-04-22T06:32:54.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.601+0000 7f9717396640 1 -- 172.21.15.44:0/4056292266 wait complete. 2024-04-22T06:32:54.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.602+0000 7f9717396640 1 Processor -- start 2024-04-22T06:32:54.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.602+0000 7f9717396640 1 -- start start 2024-04-22T06:32:54.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.602+0000 7f9717396640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97100cb320 0x7f97100c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:54.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.602+0000 7f9717396640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97100d1400 0x7f97100cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:54.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.603+0000 7f9717396640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9710155240 0x7f97101f41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:54.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.603+0000 7f9717396640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f971015bd80 con 0x7f97100cb320 2024-04-22T06:32:54.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.603+0000 7f9717396640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f971015baa0 con 0x7f97100d1400 2024-04-22T06:32:54.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.603+0000 7f9717396640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9710155620 con 0x7f9710155240 2024-04-22T06:32:54.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.603+0000 7f9715132640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97100cb320 0x7f97100c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:54.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.603+0000 7f9715132640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97100cb320 0x7f97100c9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60162/0 (socket says 172.21.15.44:60162) 2024-04-22T06:32:54.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.603+0000 7f9715132640 1 -- 172.21.15.44:0/503427170 learned_addr learned my addr 172.21.15.44:0/503427170 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:54.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.603+0000 7f9714931640 1 --2- 172.21.15.44:0/503427170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97100d1400 0x7f97100cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:54.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.603+0000 7f9715132640 1 -- 172.21.15.44:0/503427170 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9710155240 msgr2=0x7f97101f41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:54.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.603+0000 7f9715132640 1 --2- 172.21.15.44:0/503427170 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9710155240 0x7f97101f41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:54.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.603+0000 7f9715132640 1 -- 172.21.15.44:0/503427170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97100d1400 msgr2=0x7f97100cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:54.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.603+0000 7f9715132640 1 --2- 172.21.15.44:0/503427170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97100d1400 0x7f97100cc5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:54.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.603+0000 7f9715132640 1 -- 172.21.15.44:0/503427170 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97100ca8c0 con 0x7f97100cb320 2024-04-22T06:32:54.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.603+0000 7f9715132640 1 --2- 172.21.15.44:0/503427170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97100cb320 0x7f97100c9ea0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f970800c820 tx=0x7f970800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:54.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.603+0000 7f970dffb640 1 -- 172.21.15.44:0/503427170 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9708015070 con 0x7f97100cb320 2024-04-22T06:32:54.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.603+0000 7f970dffb640 1 -- 172.21.15.44:0/503427170 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9708007a30 con 0x7f97100cb320 2024-04-22T06:32:54.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.603+0000 7f9717396640 1 -- 172.21.15.44:0/503427170 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97100c7de0 con 0x7f97100cb320 2024-04-22T06:32:54.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.603+0000 7f970dffb640 1 -- 172.21.15.44:0/503427170 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9708007de0 con 0x7f97100cb320 2024-04-22T06:32:54.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.604+0000 7f9717396640 1 -- 172.21.15.44:0/503427170 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9710149690 con 0x7f97100cb320 2024-04-22T06:32:54.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.604+0000 7f9717396640 1 -- 172.21.15.44:0/503427170 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9710068cc0 con 0x7f97100cb320 2024-04-22T06:32:54.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.605+0000 7f970dffb640 1 -- 172.21.15.44:0/503427170 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9708018070 con 0x7f97100cb320 2024-04-22T06:32:54.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.605+0000 7f970dffb640 1 --2- 172.21.15.44:0/503427170 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f96fc06c8f0 0x7f96fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:54.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.605+0000 7f970dffb640 1 -- 172.21.15.44:0/503427170 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f970808be80 con 0x7f97100cb320 2024-04-22T06:32:54.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.605+0000 7f9714931640 1 --2- 172.21.15.44:0/503427170 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f96fc06c8f0 0x7f96fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:54.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.606+0000 7f9714931640 1 --2- 172.21.15.44:0/503427170 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f96fc06c8f0 0x7f96fc06edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9704036000 tx=0x7f9704008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:54.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.610+0000 7f970dffb640 1 -- 172.21.15.44:0/503427170 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9708059e20 con 0x7f97100cb320 2024-04-22T06:32:54.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.833+0000 7f9717396640 1 -- 172.21.15.44:0/503427170 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97100cadd0 con 0x7f97100cb320 2024-04-22T06:32:54.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.833+0000 7f970dffb640 1 -- 172.21.15.44:0/503427170 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+6905 (secure 0 0 0) 0x7f9708059720 con 0x7f97100cb320 2024-04-22T06:32:54.832 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:32:54.832 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/1779271860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":1779271860},{"type":"v1","addr":"172.21.15.70:6836","nonce":1779271860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T06:31:06.356312+0000","modified":"2024-04-22T06:31:56.168744+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4308,"mds_1":4305,"mds_2":4293,"mds_3":4290,"mds_4":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":4,"incarnation":17,"state":"up:active","state_seq":15,"addr":"172.21.15.44:6834/1497302315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1497302315},{"type":"v1","addr":"172.21.15.44:6834","nonce":1497302315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4290":{"gid":4290,"name":"a","rank":3,"incarnation":15,"state":"up:active","state_seq":14,"addr":"172.21.15.44:6835/2383621014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2383621014},{"type":"v1","addr":"172.21.15.44:6835","nonce":2383621014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"e","rank":2,"incarnation":13,"state":"up:active","state_seq":12,"addr":"172.21.15.44:6837/915960587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":915960587},{"type":"v1","addr":"172.21.15.44:6837","nonce":915960587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4305":{"gid":4305,"name":"b","rank":1,"incarnation":9,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6838/924955785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":924955785},{"type":"v1","addr":"172.21.15.70:6838","nonce":924955785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4308":{"gid":4308,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.70:6839/2915150040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6837","nonce":2915150040},{"type":"v1","addr":"172.21.15.70:6839","nonce":2915150040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4308,"qdb_cluster":[4287,4290,4293,4305,4308]},"id":1}]} 2024-04-22T06:32:54.832 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 18 2024-04-22T06:32:54.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.835+0000 7f9717396640 1 -- 172.21.15.44:0/503427170 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f96fc06c8f0 msgr2=0x7f96fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:54.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.835+0000 7f9717396640 1 --2- 172.21.15.44:0/503427170 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f96fc06c8f0 0x7f96fc06edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9704036000 tx=0x7f9704008030 comp rx=0 tx=0).stop 2024-04-22T06:32:54.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.835+0000 7f9717396640 1 -- 172.21.15.44:0/503427170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97100cb320 msgr2=0x7f97100c9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:54.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.835+0000 7f9717396640 1 --2- 172.21.15.44:0/503427170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97100cb320 0x7f97100c9ea0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f970800c820 tx=0x7f970800cce0 comp rx=0 tx=0).stop 2024-04-22T06:32:54.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.835+0000 7f9717396640 1 -- 172.21.15.44:0/503427170 shutdown_connections 2024-04-22T06:32:54.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.835+0000 7f9717396640 1 --2- 172.21.15.44:0/503427170 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f96fc06c8f0 0x7f96fc06edb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:54.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.835+0000 7f9717396640 1 --2- 172.21.15.44:0/503427170 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9710155240 0x7f97101f41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:54.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.835+0000 7f9717396640 1 --2- 172.21.15.44:0/503427170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97100d1400 0x7f97100cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:54.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.835+0000 7f9717396640 1 --2- 172.21.15.44:0/503427170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97100cb320 0x7f97100c9ea0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:54.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.835+0000 7f9717396640 1 -- 172.21.15.44:0/503427170 >> 172.21.15.44:0/503427170 conn(0x7f97100750d0 msgr2=0x7f97101539c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:54.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.835+0000 7f9717396640 1 -- 172.21.15.44:0/503427170 shutdown_connections 2024-04-22T06:32:54.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.835+0000 7f9717396640 1 -- 172.21.15.44:0/503427170 wait complete. 2024-04-22T06:32:54.845 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.4308 quiesce db --set-id ab138efc --release --await' 2024-04-22T06:32:54.845 DEBUG:teuthology.orchestra.run.smithi044:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4308 quiesce db --set-id ab138efc --release --await 2024-04-22T06:32:54.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.953+0000 7fd16fb0e640 1 Processor -- start 2024-04-22T06:32:54.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.953+0000 7fd16fb0e640 1 -- start start 2024-04-22T06:32:54.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.954+0000 7fd16fb0e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd168151f50 0x7fd168158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:54.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.954+0000 7fd16fb0e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd1681598a0 0x7fd168159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:54.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.954+0000 7fd16fb0e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd168158db0 con 0x7fd1680c8410 2024-04-22T06:32:54.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.954+0000 7fd16fb0e640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fd16815a220 con 0x7fd1681598a0 2024-04-22T06:32:54.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.954+0000 7fd16fb0e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fd16815b770 con 0x7fd168151f50 2024-04-22T06:32:54.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.954+0000 7fd16e0ab640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd1681598a0 0x7fd168159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:54.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.954+0000 7fd16e0ab640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd1681598a0 0x7fd168159ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60168/0 (socket says 172.21.15.44:60168) 2024-04-22T06:32:54.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.954+0000 7fd16e0ab640 1 -- 172.21.15.44:0/2391077760 learned_addr learned my addr 172.21.15.44:0/2391077760 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:54.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.954+0000 7fd16d8aa640 1 --2- 172.21.15.44:0/2391077760 >> v2:172.21.15.70:3300/0 conn(0x7fd168151f50 0x7fd168158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:54.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.955+0000 7fd16e0ab640 1 -- 172.21.15.44:0/2391077760 >> v2:172.21.15.70:3300/0 conn(0x7fd168151f50 msgr2=0x7fd168158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:54.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.955+0000 7fd16e0ab640 1 --2- 172.21.15.44:0/2391077760 >> v2:172.21.15.70:3300/0 conn(0x7fd168151f50 0x7fd168158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:54.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.955+0000 7fd16e0ab640 1 -- 172.21.15.44:0/2391077760 >> v1:172.21.15.70:6789/0 conn(0x7fd1680c8410 legacy=0x7fd1680cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:54.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.955+0000 7fd16e0ab640 1 -- 172.21.15.44:0/2391077760 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd16815ba50 con 0x7fd1681598a0 2024-04-22T06:32:54.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.955+0000 7fd16e0ab640 1 --2- 172.21.15.44:0/2391077760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1681598a0 0x7fd168159ce0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd1680c8130 tx=0x7fd15802c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1efe1755fb64224d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:54.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.956+0000 7fd16c8a8640 1 -- 172.21.15.44:0/2391077760 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15802cbb0 con 0x7fd1681598a0 2024-04-22T06:32:54.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.956+0000 7fd16c8a8640 1 -- 172.21.15.44:0/2391077760 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd15802cd50 con 0x7fd1681598a0 2024-04-22T06:32:54.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.956+0000 7fd16c8a8640 1 -- 172.21.15.44:0/2391077760 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd158034610 con 0x7fd1681598a0 2024-04-22T06:32:54.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.956+0000 7fd16fb0e640 1 -- 172.21.15.44:0/2391077760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1681598a0 msgr2=0x7fd168159ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:54.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.956+0000 7fd16fb0e640 1 --2- 172.21.15.44:0/2391077760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1681598a0 0x7fd168159ce0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd1680c8130 tx=0x7fd15802c010 comp rx=0 tx=0).stop 2024-04-22T06:32:54.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.956+0000 7fd16fb0e640 1 -- 172.21.15.44:0/2391077760 shutdown_connections 2024-04-22T06:32:54.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.956+0000 7fd16fb0e640 1 --2- 172.21.15.44:0/2391077760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1681598a0 0x7fd168159ce0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:54.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.956+0000 7fd16fb0e640 1 --2- 172.21.15.44:0/2391077760 >> v2:172.21.15.70:3300/0 conn(0x7fd168151f50 0x7fd168158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:54.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.956+0000 7fd16fb0e640 1 -- 172.21.15.44:0/2391077760 >> 172.21.15.44:0/2391077760 conn(0x7fd16806d530 msgr2=0x7fd168074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:54.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.956+0000 7fd16fb0e640 1 -- 172.21.15.44:0/2391077760 shutdown_connections 2024-04-22T06:32:54.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.956+0000 7fd16fb0e640 1 -- 172.21.15.44:0/2391077760 wait complete. 2024-04-22T06:32:54.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.957+0000 7fd16fb0e640 1 Processor -- start 2024-04-22T06:32:54.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.957+0000 7fd16fb0e640 1 -- start start 2024-04-22T06:32:54.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.958+0000 7fd16fb0e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1680c8410 0x7fd1681f24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:54.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.958+0000 7fd16fb0e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd168151f50 0x7fd1681e46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:54.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.958+0000 7fd16fb0e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd1681598a0 0x7fd168152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:54.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.958+0000 7fd16fb0e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd16815c2f0 con 0x7fd168151f50 2024-04-22T06:32:54.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.958+0000 7fd16fb0e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd16815c010 con 0x7fd1680c8410 2024-04-22T06:32:54.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.958+0000 7fd16fb0e640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd168159380 con 0x7fd1681598a0 2024-04-22T06:32:54.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.958+0000 7fd16d0a9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd168151f50 0x7fd1681e46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:54.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.958+0000 7fd16d0a9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd168151f50 0x7fd1681e46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60176/0 (socket says 172.21.15.44:60176) 2024-04-22T06:32:54.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.958+0000 7fd16d0a9640 1 -- 172.21.15.44:0/3806281978 learned_addr learned my addr 172.21.15.44:0/3806281978 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:54.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.958+0000 7fd16d8aa640 1 --2- 172.21.15.44:0/3806281978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1680c8410 0x7fd1681f24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:54.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.958+0000 7fd16e0ab640 1 --2- 172.21.15.44:0/3806281978 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd1681598a0 0x7fd168152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:54.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.958+0000 7fd16d0a9640 1 -- 172.21.15.44:0/3806281978 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd1681598a0 msgr2=0x7fd168152ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:54.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.958+0000 7fd16d0a9640 1 --2- 172.21.15.44:0/3806281978 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd1681598a0 0x7fd168152ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:54.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.958+0000 7fd16d0a9640 1 -- 172.21.15.44:0/3806281978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1680c8410 msgr2=0x7fd1681f24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:54.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.958+0000 7fd16d0a9640 1 --2- 172.21.15.44:0/3806281978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1680c8410 0x7fd1681f24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:54.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.958+0000 7fd16d0a9640 1 -- 172.21.15.44:0/3806281978 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1681ee770 con 0x7fd168151f50 2024-04-22T06:32:54.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.958+0000 7fd16d8aa640 1 --2- 172.21.15.44:0/3806281978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1680c8410 0x7fd1681f24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:32:54.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.959+0000 7fd16d0a9640 1 --2- 172.21.15.44:0/3806281978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd168151f50 0x7fd1681e46e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd15c00c3c0 tx=0x7fd15c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:54.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.959+0000 7fd1667fc640 1 -- 172.21.15.44:0/3806281978 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15c0052d0 con 0x7fd168151f50 2024-04-22T06:32:54.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.959+0000 7fd1667fc640 1 -- 172.21.15.44:0/3806281978 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd15c005490 con 0x7fd168151f50 2024-04-22T06:32:54.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.959+0000 7fd16fb0e640 1 -- 172.21.15.44:0/3806281978 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1681eea50 con 0x7fd168151f50 2024-04-22T06:32:54.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.959+0000 7fd1667fc640 1 -- 172.21.15.44:0/3806281978 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15c007260 con 0x7fd168151f50 2024-04-22T06:32:54.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.959+0000 7fd16fb0e640 1 -- 172.21.15.44:0/3806281978 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1681ef010 con 0x7fd168151f50 2024-04-22T06:32:54.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.960+0000 7fd1667fc640 1 -- 172.21.15.44:0/3806281978 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd15c007400 con 0x7fd168151f50 2024-04-22T06:32:54.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.961+0000 7fd1667fc640 1 --2- 172.21.15.44:0/3806281978 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd15406c920 0x7fd15406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:54.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.961+0000 7fd1667fc640 1 -- 172.21.15.44:0/3806281978 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fd15c08b490 con 0x7fd168151f50 2024-04-22T06:32:54.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.961+0000 7fd16d8aa640 1 --2- 172.21.15.44:0/3806281978 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd15406c920 0x7fd15406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:54.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.961+0000 7fd16d8aa640 1 --2- 172.21.15.44:0/3806281978 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd15406c920 0x7fd15406ede0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd160009b90 tx=0x7fd160008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:54.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.962+0000 7fd1727db740 1 Processor -- start 2024-04-22T06:32:54.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.962+0000 7fd1727db740 1 -- start start 2024-04-22T06:32:54.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.963+0000 7fd1727db740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d9f83fb7d0 0x55d9f83fbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:54.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.963+0000 7fd1727db740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d9f83ba1b0 0x55d9f83fc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:54.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.963+0000 7fd1727db740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9f83bcfe0 0x55d9f83bbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:54.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.963+0000 7fd1727db740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d9f8321500 con 0x55d9f83bcfe0 2024-04-22T06:32:54.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.963+0000 7fd1727db740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55d9f82c76a0 con 0x55d9f83fb7d0 2024-04-22T06:32:54.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.963+0000 7fd1727db740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55d9f831d4c0 con 0x55d9f83ba1b0 2024-04-22T06:32:54.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.963+0000 7fd16e0ab640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9f83bcfe0 0x55d9f83bbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:54.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.963+0000 7fd16e0ab640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9f83bcfe0 0x55d9f83bbbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60180/0 (socket says 172.21.15.44:60180) 2024-04-22T06:32:54.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.963+0000 7fd16e0ab640 1 -- 172.21.15.44:0/3652647336 learned_addr learned my addr 172.21.15.44:0/3652647336 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:54.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.963+0000 7fd16d8aa640 1 --2- 172.21.15.44:0/3652647336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d9f83fb7d0 0x55d9f83fbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:54.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.963+0000 7fd16d0a9640 1 --2- 172.21.15.44:0/3652647336 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d9f83ba1b0 0x55d9f83fc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:54.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.963+0000 7fd16e0ab640 1 -- 172.21.15.44:0/3652647336 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d9f83ba1b0 msgr2=0x55d9f83fc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:54.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.963+0000 7fd16e0ab640 1 --2- 172.21.15.44:0/3652647336 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d9f83ba1b0 0x55d9f83fc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:54.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.963+0000 7fd16e0ab640 1 -- 172.21.15.44:0/3652647336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d9f83fb7d0 msgr2=0x55d9f83fbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:54.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.963+0000 7fd16e0ab640 1 --2- 172.21.15.44:0/3652647336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d9f83fb7d0 0x55d9f83fbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:54.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.963+0000 7fd16e0ab640 1 -- 172.21.15.44:0/3652647336 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9f82e65e0 con 0x55d9f83bcfe0 2024-04-22T06:32:54.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.963+0000 7fd16d0a9640 1 --2- 172.21.15.44:0/3652647336 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d9f83ba1b0 0x55d9f83fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:32:54.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.963+0000 7fd16e0ab640 1 --2- 172.21.15.44:0/3652647336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9f83bcfe0 0x55d9f83bbbf0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd158009f40 tx=0x7fd15802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f32228c2c2e50f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:54.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.963+0000 7fd14f7fe640 1 -- 172.21.15.44:0/3652647336 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15802cbb0 con 0x55d9f83bcfe0 2024-04-22T06:32:54.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.963+0000 7fd14f7fe640 1 -- 172.21.15.44:0/3652647336 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd15802cd50 con 0x55d9f83bcfe0 2024-04-22T06:32:54.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.963+0000 7fd14f7fe640 1 -- 172.21.15.44:0/3652647336 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15803e630 con 0x55d9f83bcfe0 2024-04-22T06:32:54.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.964+0000 7fd1727db740 1 -- 172.21.15.44:0/3652647336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9f83bcfe0 msgr2=0x55d9f83bbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:54.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.964+0000 7fd1727db740 1 --2- 172.21.15.44:0/3652647336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9f83bcfe0 0x55d9f83bbbf0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd158009f40 tx=0x7fd15802e000 comp rx=0 tx=0).stop 2024-04-22T06:32:54.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.964+0000 7fd1727db740 1 -- 172.21.15.44:0/3652647336 shutdown_connections 2024-04-22T06:32:54.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.964+0000 7fd1727db740 1 --2- 172.21.15.44:0/3652647336 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d9f83ba1b0 0x55d9f83fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:54.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.964+0000 7fd1727db740 1 --2- 172.21.15.44:0/3652647336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9f83bcfe0 0x55d9f83bbbf0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:54.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.964+0000 7fd1727db740 1 --2- 172.21.15.44:0/3652647336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d9f83fb7d0 0x55d9f83fbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:54.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.964+0000 7fd1727db740 1 -- 172.21.15.44:0/3652647336 >> 172.21.15.44:0/3652647336 conn(0x55d9f83b8550 msgr2=0x55d9f83bae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:54.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.964+0000 7fd1727db740 1 -- 172.21.15.44:0/3652647336 shutdown_connections 2024-04-22T06:32:54.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.964+0000 7fd1727db740 1 -- 172.21.15.44:0/3652647336 wait complete. 2024-04-22T06:32:54.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.965+0000 7fd1727db740 1 -- start start 2024-04-22T06:32:54.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.965+0000 7fd1727db740 1 Processor -- start 2024-04-22T06:32:54.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.966+0000 7fd1727db740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d9f83bcfe0 0x55d9f8400290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:54.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.966+0000 7fd1727db740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d9f83fb7d0 0x55d9f8410860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:54.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.966+0000 7fd1727db740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9f8408b60 0x55d9f83bbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:54.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.966+0000 7fd1727db740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d9f83bac90 con 0x55d9f8408b60 2024-04-22T06:32:54.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.966+0000 7fd1727db740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55d9f82daa60 con 0x55d9f83fb7d0 2024-04-22T06:32:54.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.966+0000 7fd1727db740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55d9f8409fc0 con 0x55d9f83bcfe0 2024-04-22T06:32:54.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.966+0000 7fd1727db740 10 client.0 _ll_register_callbacks cb 0x55d9f8033110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T06:32:54.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.966+0000 7fd16e0ab640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9f8408b60 0x55d9f83bbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:54.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.966+0000 7fd16e0ab640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9f8408b60 0x55d9f83bbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60182/0 (socket says 172.21.15.44:60182) 2024-04-22T06:32:54.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.966+0000 7fd16e0ab640 1 -- 172.21.15.44:0/4117507063 learned_addr learned my addr 172.21.15.44:0/4117507063 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:54.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.966+0000 7fd16d0a9640 1 --2- 172.21.15.44:0/4117507063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d9f83fb7d0 0x55d9f8410860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:54.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.966+0000 7fd16d8aa640 1 --2- 172.21.15.44:0/4117507063 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d9f83bcfe0 0x55d9f8400290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:54.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.966+0000 7fd16e0ab640 1 -- 172.21.15.44:0/4117507063 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d9f83bcfe0 msgr2=0x55d9f8400290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:54.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.966+0000 7fd16e0ab640 1 --2- 172.21.15.44:0/4117507063 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d9f83bcfe0 0x55d9f8400290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:54.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.966+0000 7fd16e0ab640 1 -- 172.21.15.44:0/4117507063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d9f83fb7d0 msgr2=0x55d9f8410860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:54.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.966+0000 7fd16e0ab640 1 --2- 172.21.15.44:0/4117507063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d9f83fb7d0 0x55d9f8410860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:54.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.966+0000 7fd16e0ab640 1 -- 172.21.15.44:0/4117507063 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9f8033920 con 0x55d9f8408b60 2024-04-22T06:32:54.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.966+0000 7fd16e0ab640 1 --2- 172.21.15.44:0/4117507063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9f8408b60 0x55d9f83bbbd0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd158009f10 tx=0x7fd158004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c2f973ae0263ccd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:54.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.966+0000 7fd14cff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T06:32:54.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.966+0000 7fd14cff9640 1 -- 172.21.15.44:0/4117507063 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15803eb80 con 0x55d9f8408b60 2024-04-22T06:32:54.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.966+0000 7fd14cff9640 1 -- 172.21.15.44:0/4117507063 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fd15803ed20 con 0x55d9f8408b60 2024-04-22T06:32:54.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.966+0000 7fd1727db740 1 -- 172.21.15.44:0/4117507063 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd15802cd50 con 0x55d9f8408b60 2024-04-22T06:32:54.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.967+0000 7fd14cff9640 1 -- 172.21.15.44:0/4117507063 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fd158055050 con 0x55d9f8408b60 2024-04-22T06:32:54.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.967+0000 7fd1727db740 10 client.5313 fetch_fsmap learned FSMap version 18 2024-04-22T06:32:54.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.967+0000 7fd1727db740 1 -- 172.21.15.44:0/4117507063 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55d9f80ffd30 con 0x55d9f8408b60 2024-04-22T06:32:54.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.967+0000 7fd14cff9640 1 -- 172.21.15.44:0/4117507063 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd15802cd50 con 0x55d9f8408b60 2024-04-22T06:32:54.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.967+0000 7fd14cff9640 1 -- 172.21.15.44:0/4117507063 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 18) ==== 4792+0+0 (secure 0 0 0) 0x7fd158058020 con 0x55d9f8408b60 2024-04-22T06:32:54.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.967+0000 7fd1727db740 10 client.5313 fetch_fsmap finished waiting for FSMap version 18 2024-04-22T06:32:54.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.967+0000 7fd1727db740 10 client.5313 resolve_mds: validated gid 4308 aka daemon mds.d 2024-04-22T06:32:54.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.967+0000 7fd1727db740 20 client.5313 populate_metadata read hostname 'smithi044' 2024-04-22T06:32:54.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.967+0000 7fd1727db740 1 --2- 172.21.15.44:0/4117507063 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x55d9f840a1c0 0x55d9f840a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:54.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.967+0000 7fd1727db740 4 client.5313 mds_command: new command op to 4308 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T06:32:54.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.967+0000 7fd1727db740 1 -- 172.21.15.44:0/4117507063 --> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9f82f4a10 con 0x55d9f840a1c0 2024-04-22T06:32:54.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.967+0000 7fd16d8aa640 1 --2- 172.21.15.44:0/4117507063 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x55d9f840a1c0 0x55d9f840a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:54.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.968+0000 7fd14cff9640 10 client.5313 ms_handle_connect on v2:172.21.15.70:6837/2915150040 2024-04-22T06:32:54.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.968+0000 7fd16d8aa640 1 --2- 172.21.15.44:0/4117507063 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x55d9f840a1c0 0x55d9f840a5d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5fcba3e4e5e475f server_cookie=6cdd7d2ccf162fa in_seq=0 out_seq=0 2024-04-22T06:32:54.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.968+0000 7fd14cff9640 1 -- 172.21.15.44:0/4117507063 <== mds.0 v2:172.21.15.70:6837/2915150040 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d9f82f4a10 con 0x55d9f840a1c0 2024-04-22T06:32:54.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.968+0000 7fd14cff9640 10 client.5313 handle_command_reply: tid=0 2024-04-22T06:32:54.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.968+0000 7fd1727db740 1 client.5313 shutdown 2024-04-22T06:32:54.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.968+0000 7fd1727db740 1 -- 172.21.15.44:0/4117507063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9f8408b60 msgr2=0x55d9f83bbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:54.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.968+0000 7fd1727db740 1 --2- 172.21.15.44:0/4117507063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9f8408b60 0x55d9f83bbbd0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd158009f10 tx=0x7fd158004ab0 comp rx=0 tx=0).stop 2024-04-22T06:32:54.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.968+0000 7fd1727db740 1 -- 172.21.15.44:0/4117507063 shutdown_connections 2024-04-22T06:32:54.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.968+0000 7fd1727db740 1 --2- 172.21.15.44:0/4117507063 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x55d9f840a1c0 0x55d9f840a5d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:54.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.968+0000 7fd1727db740 1 --2- 172.21.15.44:0/4117507063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9f8408b60 0x55d9f83bbbd0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:54.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.968+0000 7fd1727db740 1 --2- 172.21.15.44:0/4117507063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d9f83fb7d0 0x55d9f8410860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:54.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.968+0000 7fd1727db740 1 --2- 172.21.15.44:0/4117507063 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d9f83bcfe0 0x55d9f8400290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:54.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.968+0000 7fd14cff9640 0 client.5313 ms_handle_reset on v2:172.21.15.70:6837/2915150040 2024-04-22T06:32:54.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.968+0000 7fd1727db740 1 -- 172.21.15.44:0/4117507063 >> 172.21.15.44:0/4117507063 conn(0x55d9f83ba1b0 msgr2=0x55d9f83e5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:54.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.969+0000 7fd1727db740 1 -- 172.21.15.44:0/4117507063 shutdown_connections 2024-04-22T06:32:54.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.969+0000 7fd1727db740 1 -- 172.21.15.44:0/4117507063 wait complete. 2024-04-22T06:32:54.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.969+0000 7fd1727db740 20 client.5313 trim_cache size 0 max 16384 2024-04-22T06:32:54.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.986+0000 7fd1727db740 1 Processor -- start 2024-04-22T06:32:54.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.986+0000 7fd1727db740 1 -- start start 2024-04-22T06:32:54.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.987+0000 7fd1727db740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d9f83bcfe0 0x55d9f83b5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:54.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.987+0000 7fd1727db740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9f83fb7d0 0x55d9f83b57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:54.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.987+0000 7fd1727db740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d9f8408b60 0x55d9f8424440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:54.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.987+0000 7fd1727db740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd1681fb900 con 0x55d9f83fb7d0 2024-04-22T06:32:54.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.987+0000 7fd1727db740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd1681f9dc0 con 0x55d9f8408b60 2024-04-22T06:32:54.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.987+0000 7fd1727db740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd1681fb230 con 0x55d9f83bcfe0 2024-04-22T06:32:54.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.987+0000 7fd16d0a9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9f83fb7d0 0x55d9f83b57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:54.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.987+0000 7fd16d0a9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9f83fb7d0 0x55d9f83b57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60192/0 (socket says 172.21.15.44:60192) 2024-04-22T06:32:54.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.987+0000 7fd16d0a9640 1 -- 172.21.15.44:0/2966325388 learned_addr learned my addr 172.21.15.44:0/2966325388 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:54.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.987+0000 7fd16d8aa640 1 --2- 172.21.15.44:0/2966325388 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d9f83bcfe0 0x55d9f83b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:54.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.987+0000 7fd16d0a9640 1 -- 172.21.15.44:0/2966325388 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d9f83bcfe0 msgr2=0x55d9f83b5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:54.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.987+0000 7fd16d0a9640 1 --2- 172.21.15.44:0/2966325388 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d9f83bcfe0 0x55d9f83b5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:54.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.987+0000 7fd16e0ab640 1 --2- 172.21.15.44:0/2966325388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d9f8408b60 0x55d9f8424440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:54.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.987+0000 7fd16d0a9640 1 -- 172.21.15.44:0/2966325388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d9f8408b60 msgr2=0x55d9f8424440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:54.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.987+0000 7fd16d0a9640 1 --2- 172.21.15.44:0/2966325388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d9f8408b60 0x55d9f8424440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:54.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.987+0000 7fd16d0a9640 1 -- 172.21.15.44:0/2966325388 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9f83bbf40 con 0x55d9f83fb7d0 2024-04-22T06:32:54.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.987+0000 7fd16d0a9640 1 --2- 172.21.15.44:0/2966325388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9f83fb7d0 0x55d9f83b57b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd15c058c70 tx=0x7fd15c08b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b89123a69f358dc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:54.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.987+0000 7fd14effd640 1 -- 172.21.15.44:0/2966325388 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15c0619a0 con 0x55d9f83fb7d0 2024-04-22T06:32:54.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.987+0000 7fd14effd640 1 -- 172.21.15.44:0/2966325388 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd15c087070 con 0x55d9f83fb7d0 2024-04-22T06:32:54.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.987+0000 7fd14effd640 1 -- 172.21.15.44:0/2966325388 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15c0553a0 con 0x55d9f83fb7d0 2024-04-22T06:32:54.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.988+0000 7fd1727db740 1 -- 172.21.15.44:0/2966325388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9f83fb7d0 msgr2=0x55d9f83b57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:32:54.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.988+0000 7fd1727db740 1 --2- 172.21.15.44:0/2966325388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9f83fb7d0 0x55d9f83b57b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd15c058c70 tx=0x7fd15c08b040 comp rx=0 tx=0).stop 2024-04-22T06:32:54.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.988+0000 7fd1727db740 1 -- 172.21.15.44:0/2966325388 shutdown_connections 2024-04-22T06:32:54.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.988+0000 7fd1727db740 1 --2- 172.21.15.44:0/2966325388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d9f8408b60 0x55d9f8424440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:54.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.988+0000 7fd1727db740 1 --2- 172.21.15.44:0/2966325388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9f83fb7d0 0x55d9f83b57b0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:54.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.988+0000 7fd1727db740 1 --2- 172.21.15.44:0/2966325388 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d9f83bcfe0 0x55d9f83b5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:54.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.988+0000 7fd1727db740 1 -- 172.21.15.44:0/2966325388 >> 172.21.15.44:0/2966325388 conn(0x55d9f83ba1b0 msgr2=0x55d9f83bae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:32:54.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.988+0000 7fd1727db740 1 -- 172.21.15.44:0/2966325388 shutdown_connections 2024-04-22T06:32:54.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.988+0000 7fd1727db740 1 -- 172.21.15.44:0/2966325388 wait complete. 2024-04-22T06:32:54.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.989+0000 7fd1727db740 1 -- start start 2024-04-22T06:32:54.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.989+0000 7fd1727db740 1 Processor -- start 2024-04-22T06:32:54.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.989+0000 7fd1727db740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d9f83bcfe0 0x55d9f83d88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:54.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.989+0000 7fd1727db740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9f83fb7d0 0x55d9f8477460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:54.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.989+0000 7fd1727db740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d9f8408b60 0x55d9f84779a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:54.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.989+0000 7fd1727db740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd1681fab60 con 0x55d9f83fb7d0 2024-04-22T06:32:54.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.989+0000 7fd1727db740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd1681fa490 con 0x55d9f83bcfe0 2024-04-22T06:32:54.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.989+0000 7fd1727db740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd1681e41b0 con 0x55d9f8408b60 2024-04-22T06:32:54.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.990+0000 7fd1727db740 10 client.0 _ll_register_callbacks cb 0x7fd1600098c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T06:32:54.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.990+0000 7fd16d0a9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9f83fb7d0 0x55d9f8477460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:54.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.990+0000 7fd16d0a9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9f83fb7d0 0x55d9f8477460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60208/0 (socket says 172.21.15.44:60208) 2024-04-22T06:32:54.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.990+0000 7fd16d0a9640 1 -- 172.21.15.44:0/3200189522 learned_addr learned my addr 172.21.15.44:0/3200189522 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:32:54.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.990+0000 7fd16d8aa640 1 --2- 172.21.15.44:0/3200189522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d9f83bcfe0 0x55d9f83d88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:54.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.990+0000 7fd16e0ab640 1 --2- 172.21.15.44:0/3200189522 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d9f8408b60 0x55d9f84779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:54.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.990+0000 7fd16d0a9640 1 -- 172.21.15.44:0/3200189522 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d9f8408b60 msgr2=0x55d9f84779a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:54.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.990+0000 7fd16d0a9640 1 --2- 172.21.15.44:0/3200189522 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d9f8408b60 0x55d9f84779a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:54.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.990+0000 7fd16d0a9640 1 -- 172.21.15.44:0/3200189522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d9f83bcfe0 msgr2=0x55d9f83d88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:32:54.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.990+0000 7fd16d0a9640 1 --2- 172.21.15.44:0/3200189522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d9f83bcfe0 0x55d9f83d88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:32:54.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.990+0000 7fd16d0a9640 1 -- 172.21.15.44:0/3200189522 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9f840a1f0 con 0x55d9f83fb7d0 2024-04-22T06:32:54.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.990+0000 7fd16d8aa640 1 --2- 172.21.15.44:0/3200189522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d9f83bcfe0 0x55d9f83d88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:32:54.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.990+0000 7fd16d0a9640 1 --2- 172.21.15.44:0/3200189522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9f83fb7d0 0x55d9f8477460 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd15c00cfd0 tx=0x7fd15c00a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=493ee5c25bc24374 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:32:54.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.990+0000 7fd14cff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T06:32:54.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.990+0000 7fd14cff9640 1 -- 172.21.15.44:0/3200189522 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15c078350 con 0x55d9f83fb7d0 2024-04-22T06:32:54.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.990+0000 7fd14cff9640 1 -- 172.21.15.44:0/3200189522 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fd15c061510 con 0x55d9f83fb7d0 2024-04-22T06:32:54.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.990+0000 7fd1727db740 1 -- 172.21.15.44:0/3200189522 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd15c087070 con 0x55d9f83fb7d0 2024-04-22T06:32:54.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.990+0000 7fd14cff9640 1 -- 172.21.15.44:0/3200189522 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fd15c07e050 con 0x55d9f83fb7d0 2024-04-22T06:32:54.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.990+0000 7fd1727db740 10 client.5319 fetch_fsmap learned FSMap version 18 2024-04-22T06:32:54.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.990+0000 7fd1727db740 1 -- 172.21.15.44:0/3200189522 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55d9f83b1850 con 0x55d9f83fb7d0 2024-04-22T06:32:54.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.990+0000 7fd14cff9640 1 -- 172.21.15.44:0/3200189522 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd15c087070 con 0x55d9f83fb7d0 2024-04-22T06:32:54.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.991+0000 7fd14cff9640 1 -- 172.21.15.44:0/3200189522 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 18) ==== 4792+0+0 (secure 0 0 0) 0x7fd15c081020 con 0x55d9f83fb7d0 2024-04-22T06:32:54.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.991+0000 7fd1727db740 10 client.5319 fetch_fsmap finished waiting for FSMap version 18 2024-04-22T06:32:54.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.991+0000 7fd1727db740 10 client.5319 resolve_mds: validated gid 4308 aka daemon mds.d 2024-04-22T06:32:54.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.991+0000 7fd1727db740 20 client.5319 populate_metadata read hostname 'smithi044' 2024-04-22T06:32:54.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.991+0000 7fd1727db740 1 --2- 172.21.15.44:0/3200189522 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x55d9f840a8f0 0x55d9f8423c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:32:54.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.991+0000 7fd1727db740 4 client.5319 mds_command: new command op to 4308 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ab138efc", "release": true, "await": true}] 2024-04-22T06:32:54.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.991+0000 7fd1727db740 1 -- 172.21.15.44:0/3200189522 --> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ab138efc", "release": true, "await": true}) -- 0x55d9f832ef20 con 0x55d9f840a8f0 2024-04-22T06:32:54.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.991+0000 7fd16d8aa640 1 --2- 172.21.15.44:0/3200189522 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x55d9f840a8f0 0x55d9f8423c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:32:54.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.991+0000 7fd16d8aa640 1 --2- 172.21.15.44:0/3200189522 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x55d9f840a8f0 0x55d9f8423c30 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2f7f5147e1a1060 server_cookie=be76db2e6086f780 in_seq=0 out_seq=0 2024-04-22T06:32:54.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:32:54.991+0000 7fd14cff9640 10 client.5319 ms_handle_connect on v2:172.21.15.70:6837/2915150040 2024-04-22T06:32:54.995 INFO:tasks.ceph.mds.a.smithi044.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7f95e3e18640 time 2024-04-22T06:32:54.992849+0000 2024-04-22T06:32:54.996 INFO:tasks.ceph.mds.a.smithi044.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T06:32:54.996 INFO:tasks.ceph.mds.c.smithi044.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7f7de9fa4640 time 2024-04-22T06:32:54.992905+0000 2024-04-22T06:32:54.996 INFO:tasks.ceph.mds.c.smithi044.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T06:32:54.996 INFO:tasks.ceph.mds.e.smithi044.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7f5a98b76640 time 2024-04-22T06:32:54.992922+0000 2024-04-22T06:32:54.996 INFO:tasks.ceph.mds.e.smithi044.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T06:32:54.997 INFO:tasks.ceph.mds.c.smithi044.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T06:32:54.997 INFO:tasks.ceph.mds.c.smithi044.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f7df7b81f5c] 2024-04-22T06:32:54.997 INFO:tasks.ceph.mds.c.smithi044.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f7df7b8211a] 2024-04-22T06:32:54.997 INFO:tasks.ceph.mds.c.smithi044.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x555a9dea7fd2] 2024-04-22T06:32:54.997 INFO:tasks.ceph.mds.c.smithi044.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x555a9dea8091] 2024-04-22T06:32:54.997 INFO:tasks.ceph.mds.c.smithi044.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x555a9de62a96] 2024-04-22T06:32:54.997 INFO:tasks.ceph.mds.c.smithi044.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x555a9de629f6] 2024-04-22T06:32:54.997 INFO:tasks.ceph.mds.c.smithi044.stderr: 7: ceph-mds(+0x48e3cf) [0x555a9e0003cf] 2024-04-22T06:32:54.998 INFO:tasks.ceph.mds.c.smithi044.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x555a9e050c1b] 2024-04-22T06:32:54.998 INFO:tasks.ceph.mds.c.smithi044.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7f7df7289c02] 2024-04-22T06:32:54.998 INFO:tasks.ceph.mds.c.smithi044.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7f7df730ec40] 2024-04-22T06:32:54.998 INFO:tasks.ceph.mds.c.smithi044.stderr:*** Caught signal (Aborted) ** 2024-04-22T06:32:54.998 INFO:tasks.ceph.mds.c.smithi044.stderr: in thread 7f7de9fa4640 thread_name:quiesce.agt 2024-04-22T06:32:54.998 INFO:tasks.ceph.mds.c.smithi044.stderr:2024-04-22T06:32:54.992+0000 7f7de9fa4640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7f7de9fa4640 time 2024-04-22T06:32:54.992905+0000 2024-04-22T06:32:54.998 INFO:tasks.ceph.mds.c.smithi044.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T06:32:54.998 INFO:tasks.ceph.mds.c.smithi044.stderr: 2024-04-22T06:32:54.998 INFO:tasks.ceph.mds.c.smithi044.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T06:32:54.998 INFO:tasks.ceph.mds.c.smithi044.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f7df7b81f5c] 2024-04-22T06:32:54.998 INFO:tasks.ceph.mds.c.smithi044.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f7df7b8211a] 2024-04-22T06:32:54.998 INFO:tasks.ceph.mds.c.smithi044.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x555a9dea7fd2] 2024-04-22T06:32:54.998 INFO:tasks.ceph.mds.c.smithi044.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x555a9dea8091] 2024-04-22T06:32:54.999 INFO:tasks.ceph.mds.c.smithi044.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x555a9de62a96] 2024-04-22T06:32:54.999 INFO:tasks.ceph.mds.c.smithi044.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x555a9de629f6] 2024-04-22T06:32:54.999 INFO:tasks.ceph.mds.c.smithi044.stderr: 7: ceph-mds(+0x48e3cf) [0x555a9e0003cf] 2024-04-22T06:32:54.999 INFO:tasks.ceph.mds.c.smithi044.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x555a9e050c1b] 2024-04-22T06:32:54.999 INFO:tasks.ceph.mds.c.smithi044.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7f7df7289c02] 2024-04-22T06:32:54.999 INFO:tasks.ceph.mds.c.smithi044.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7f7df730ec40] 2024-04-22T06:32:54.999 INFO:tasks.ceph.mds.c.smithi044.stderr: 2024-04-22T06:32:54.999 INFO:tasks.ceph.mds.e.smithi044.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T06:32:54.999 INFO:tasks.ceph.mds.e.smithi044.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f5aa6781f5c] 2024-04-22T06:32:54.999 INFO:tasks.ceph.mds.e.smithi044.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f5aa678211a] 2024-04-22T06:32:54.999 INFO:tasks.ceph.mds.e.smithi044.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x55ffb5ae9fd2] 2024-04-22T06:32:55.000 INFO:tasks.ceph.mds.e.smithi044.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x55ffb5aea091] 2024-04-22T06:32:55.000 INFO:tasks.ceph.mds.e.smithi044.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x55ffb5aa4a96] 2024-04-22T06:32:55.000 INFO:tasks.ceph.mds.e.smithi044.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x55ffb5aa49f6] 2024-04-22T06:32:55.000 INFO:tasks.ceph.mds.e.smithi044.stderr: 7: ceph-mds(+0x48e3cf) [0x55ffb5c423cf] 2024-04-22T06:32:55.000 INFO:tasks.ceph.mds.e.smithi044.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x55ffb5c92c1b] 2024-04-22T06:32:55.000 INFO:tasks.ceph.mds.e.smithi044.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7f5aa5e89c02] 2024-04-22T06:32:55.000 INFO:tasks.ceph.mds.e.smithi044.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7f5aa5f0ec40] 2024-04-22T06:32:55.000 INFO:tasks.ceph.mds.e.smithi044.stderr:2024-04-22T06:32:54.992+0000 7f5a98b76640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7f5a98b76640 time 2024-04-22T06:32:54.992922+0000 2024-04-22T06:32:55.000 INFO:tasks.ceph.mds.e.smithi044.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T06:32:55.000 INFO:tasks.ceph.mds.e.smithi044.stderr: 2024-04-22T06:32:55.000 INFO:tasks.ceph.mds.e.smithi044.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T06:32:55.000 INFO:tasks.ceph.mds.e.smithi044.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f5aa6781f5c] 2024-04-22T06:32:55.000 INFO:tasks.ceph.mds.e.smithi044.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f5aa678211a] 2024-04-22T06:32:55.001 INFO:tasks.ceph.mds.e.smithi044.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x55ffb5ae9fd2] 2024-04-22T06:32:55.001 INFO:tasks.ceph.mds.e.smithi044.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x55ffb5aea091] 2024-04-22T06:32:55.001 INFO:tasks.ceph.mds.e.smithi044.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x55ffb5aa4a96] 2024-04-22T06:32:55.001 INFO:tasks.ceph.mds.e.smithi044.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x55ffb5aa49f6] 2024-04-22T06:32:55.001 INFO:tasks.ceph.mds.e.smithi044.stderr: 7: ceph-mds(+0x48e3cf) [0x55ffb5c423cf] 2024-04-22T06:32:55.001 INFO:tasks.ceph.mds.e.smithi044.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x55ffb5c92c1b] 2024-04-22T06:32:55.001 INFO:tasks.ceph.mds.e.smithi044.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7f5aa5e89c02] 2024-04-22T06:32:55.001 INFO:tasks.ceph.mds.e.smithi044.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7f5aa5f0ec40] 2024-04-22T06:32:55.001 INFO:tasks.ceph.mds.e.smithi044.stderr: 2024-04-22T06:32:55.001 INFO:tasks.ceph.mds.e.smithi044.stderr:*** Caught signal (Aborted) ** 2024-04-22T06:32:55.001 INFO:tasks.ceph.mds.e.smithi044.stderr: in thread 7f5a98b76640 thread_name:quiesce.agt 2024-04-22T06:32:55.002 INFO:tasks.ceph.mds.a.smithi044.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T06:32:55.002 INFO:tasks.ceph.mds.a.smithi044.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f95f1981f5c] 2024-04-22T06:32:55.002 INFO:tasks.ceph.mds.a.smithi044.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f95f198211a] 2024-04-22T06:32:55.002 INFO:tasks.ceph.mds.a.smithi044.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x5597e9caffd2] 2024-04-22T06:32:55.002 INFO:tasks.ceph.mds.a.smithi044.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x5597e9cb0091] 2024-04-22T06:32:55.002 INFO:tasks.ceph.mds.a.smithi044.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x5597e9c6aa96] 2024-04-22T06:32:55.002 INFO:tasks.ceph.mds.a.smithi044.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x5597e9c6a9f6] 2024-04-22T06:32:55.002 INFO:tasks.ceph.mds.a.smithi044.stderr: 7: ceph-mds(+0x48e3cf) [0x5597e9e083cf] 2024-04-22T06:32:55.002 INFO:tasks.ceph.mds.a.smithi044.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x5597e9e58c1b] 2024-04-22T06:32:55.002 INFO:tasks.ceph.mds.a.smithi044.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7f95f1089c02] 2024-04-22T06:32:55.002 INFO:tasks.ceph.mds.a.smithi044.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7f95f110ec40] 2024-04-22T06:32:55.002 INFO:tasks.ceph.mds.a.smithi044.stderr:*** Caught signal (Aborted) ** 2024-04-22T06:32:55.003 INFO:tasks.ceph.mds.a.smithi044.stderr: in thread 7f95e3e18640 thread_name:quiesce.agt 2024-04-22T06:32:55.003 INFO:tasks.ceph.mds.a.smithi044.stderr:2024-04-22T06:32:54.992+0000 7f95e3e18640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7f95e3e18640 time 2024-04-22T06:32:54.992849+0000 2024-04-22T06:32:55.003 INFO:tasks.ceph.mds.a.smithi044.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T06:32:55.003 INFO:tasks.ceph.mds.a.smithi044.stderr: 2024-04-22T06:32:55.003 INFO:tasks.ceph.mds.a.smithi044.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T06:32:55.003 INFO:tasks.ceph.mds.a.smithi044.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f95f1981f5c] 2024-04-22T06:32:55.003 INFO:tasks.ceph.mds.a.smithi044.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f95f198211a] 2024-04-22T06:32:55.003 INFO:tasks.ceph.mds.a.smithi044.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x5597e9caffd2] 2024-04-22T06:32:55.003 INFO:tasks.ceph.mds.a.smithi044.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x5597e9cb0091] 2024-04-22T06:32:55.003 INFO:tasks.ceph.mds.a.smithi044.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x5597e9c6aa96] 2024-04-22T06:32:55.003 INFO:tasks.ceph.mds.a.smithi044.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x5597e9c6a9f6] 2024-04-22T06:32:55.003 INFO:tasks.ceph.mds.a.smithi044.stderr: 7: ceph-mds(+0x48e3cf) [0x5597e9e083cf] 2024-04-22T06:32:55.003 INFO:tasks.ceph.mds.a.smithi044.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x5597e9e58c1b] 2024-04-22T06:32:55.003 INFO:tasks.ceph.mds.a.smithi044.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7f95f1089c02] 2024-04-22T06:32:55.003 INFO:tasks.ceph.mds.a.smithi044.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7f95f110ec40] 2024-04-22T06:32:55.004 INFO:tasks.ceph.mds.a.smithi044.stderr: 2024-04-22T06:32:55.004 INFO:tasks.ceph.mds.e.smithi044.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T06:32:55.004 INFO:tasks.ceph.mds.e.smithi044.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f5aa5e3e6f0] 2024-04-22T06:32:55.004 INFO:tasks.ceph.mds.e.smithi044.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f5aa5e8b94c] 2024-04-22T06:32:55.004 INFO:tasks.ceph.mds.e.smithi044.stderr: 3: raise() 2024-04-22T06:32:55.004 INFO:tasks.ceph.mds.e.smithi044.stderr: 4: abort() 2024-04-22T06:32:55.004 INFO:tasks.ceph.mds.e.smithi044.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f5aa6781fb6] 2024-04-22T06:32:55.004 INFO:tasks.ceph.mds.e.smithi044.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f5aa678211a] 2024-04-22T06:32:55.004 INFO:tasks.ceph.mds.e.smithi044.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x55ffb5ae9fd2] 2024-04-22T06:32:55.005 INFO:tasks.ceph.mds.e.smithi044.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x55ffb5aea091] 2024-04-22T06:32:55.005 INFO:tasks.ceph.mds.e.smithi044.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x55ffb5aa4a96] 2024-04-22T06:32:55.005 INFO:tasks.ceph.mds.e.smithi044.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x55ffb5aa49f6] 2024-04-22T06:32:55.005 INFO:tasks.ceph.mds.e.smithi044.stderr: 11: ceph-mds(+0x48e3cf) [0x55ffb5c423cf] 2024-04-22T06:32:55.005 INFO:tasks.ceph.mds.e.smithi044.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x55ffb5c92c1b] 2024-04-22T06:32:55.005 INFO:tasks.ceph.mds.e.smithi044.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7f5aa5e89c02] 2024-04-22T06:32:55.005 INFO:tasks.ceph.mds.e.smithi044.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7f5aa5f0ec40] 2024-04-22T06:32:55.005 INFO:tasks.ceph.mds.c.smithi044.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T06:32:55.005 INFO:tasks.ceph.mds.c.smithi044.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f7df723e6f0] 2024-04-22T06:32:55.005 INFO:tasks.ceph.mds.c.smithi044.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f7df728b94c] 2024-04-22T06:32:55.006 INFO:tasks.ceph.mds.c.smithi044.stderr: 3: raise() 2024-04-22T06:32:55.006 INFO:tasks.ceph.mds.c.smithi044.stderr: 4: abort() 2024-04-22T06:32:55.006 INFO:tasks.ceph.mds.c.smithi044.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f7df7b81fb6] 2024-04-22T06:32:55.006 INFO:tasks.ceph.mds.c.smithi044.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f7df7b8211a] 2024-04-22T06:32:55.006 INFO:tasks.ceph.mds.c.smithi044.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x555a9dea7fd2] 2024-04-22T06:32:55.006 INFO:tasks.ceph.mds.c.smithi044.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x555a9dea8091] 2024-04-22T06:32:55.006 INFO:tasks.ceph.mds.c.smithi044.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x555a9de62a96] 2024-04-22T06:32:55.006 INFO:tasks.ceph.mds.c.smithi044.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x555a9de629f6] 2024-04-22T06:32:55.006 INFO:tasks.ceph.mds.c.smithi044.stderr: 11: ceph-mds(+0x48e3cf) [0x555a9e0003cf] 2024-04-22T06:32:55.006 INFO:tasks.ceph.mds.c.smithi044.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x555a9e050c1b] 2024-04-22T06:32:55.006 INFO:tasks.ceph.mds.c.smithi044.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7f7df7289c02] 2024-04-22T06:32:55.006 INFO:tasks.ceph.mds.c.smithi044.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7f7df730ec40] 2024-04-22T06:32:55.007 INFO:tasks.ceph.mds.a.smithi044.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T06:32:55.007 INFO:tasks.ceph.mds.a.smithi044.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f95f103e6f0] 2024-04-22T06:32:55.007 INFO:tasks.ceph.mds.a.smithi044.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f95f108b94c] 2024-04-22T06:32:55.007 INFO:tasks.ceph.mds.a.smithi044.stderr: 3: raise() 2024-04-22T06:32:55.007 INFO:tasks.ceph.mds.a.smithi044.stderr: 4: abort() 2024-04-22T06:32:55.007 INFO:tasks.ceph.mds.a.smithi044.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f95f1981fb6] 2024-04-22T06:32:55.007 INFO:tasks.ceph.mds.a.smithi044.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f95f198211a] 2024-04-22T06:32:55.007 INFO:tasks.ceph.mds.a.smithi044.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x5597e9caffd2] 2024-04-22T06:32:55.007 INFO:tasks.ceph.mds.a.smithi044.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x5597e9cb0091] 2024-04-22T06:32:55.007 INFO:tasks.ceph.mds.a.smithi044.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x5597e9c6aa96] 2024-04-22T06:32:55.007 INFO:tasks.ceph.mds.a.smithi044.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x5597e9c6a9f6] 2024-04-22T06:32:55.007 INFO:tasks.ceph.mds.a.smithi044.stderr: 11: ceph-mds(+0x48e3cf) [0x5597e9e083cf] 2024-04-22T06:32:55.008 INFO:tasks.ceph.mds.a.smithi044.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x5597e9e58c1b] 2024-04-22T06:32:55.008 INFO:tasks.ceph.mds.a.smithi044.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7f95f1089c02] 2024-04-22T06:32:55.008 INFO:tasks.ceph.mds.a.smithi044.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7f95f110ec40] 2024-04-22T06:32:55.008 INFO:tasks.ceph.mds.c.smithi044.stderr:2024-04-22T06:32:54.993+0000 7f7de9fa4640 -1 *** Caught signal (Aborted) ** 2024-04-22T06:32:55.008 INFO:tasks.ceph.mds.c.smithi044.stderr: in thread 7f7de9fa4640 thread_name:quiesce.agt 2024-04-22T06:32:55.008 INFO:tasks.ceph.mds.c.smithi044.stderr: 2024-04-22T06:32:55.008 INFO:tasks.ceph.mds.c.smithi044.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T06:32:55.008 INFO:tasks.ceph.mds.c.smithi044.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f7df723e6f0] 2024-04-22T06:32:55.008 INFO:tasks.ceph.mds.c.smithi044.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f7df728b94c] 2024-04-22T06:32:55.008 INFO:tasks.ceph.mds.c.smithi044.stderr: 3: raise() 2024-04-22T06:32:55.009 INFO:tasks.ceph.mds.c.smithi044.stderr: 4: abort() 2024-04-22T06:32:55.009 INFO:tasks.ceph.mds.c.smithi044.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f7df7b81fb6] 2024-04-22T06:32:55.009 INFO:tasks.ceph.mds.c.smithi044.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f7df7b8211a] 2024-04-22T06:32:55.009 INFO:tasks.ceph.mds.c.smithi044.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x555a9dea7fd2] 2024-04-22T06:32:55.009 INFO:tasks.ceph.mds.c.smithi044.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x555a9dea8091] 2024-04-22T06:32:55.009 INFO:tasks.ceph.mds.c.smithi044.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x555a9de62a96] 2024-04-22T06:32:55.009 INFO:tasks.ceph.mds.c.smithi044.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x555a9de629f6] 2024-04-22T06:32:55.009 INFO:tasks.ceph.mds.c.smithi044.stderr: 11: ceph-mds(+0x48e3cf) [0x555a9e0003cf] 2024-04-22T06:32:55.009 INFO:tasks.ceph.mds.c.smithi044.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x555a9e050c1b] 2024-04-22T06:32:55.009 INFO:tasks.ceph.mds.c.smithi044.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7f7df7289c02] 2024-04-22T06:32:55.009 INFO:tasks.ceph.mds.c.smithi044.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7f7df730ec40] 2024-04-22T06:32:55.009 INFO:tasks.ceph.mds.c.smithi044.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T06:32:55.009 INFO:tasks.ceph.mds.c.smithi044.stderr: 2024-04-22T06:32:55.010 INFO:tasks.ceph.mds.a.smithi044.stderr:2024-04-22T06:32:54.993+0000 7f95e3e18640 -1 *** Caught signal (Aborted) ** 2024-04-22T06:32:55.010 INFO:tasks.ceph.mds.a.smithi044.stderr: in thread 7f95e3e18640 thread_name:quiesce.agt 2024-04-22T06:32:55.010 INFO:tasks.ceph.mds.a.smithi044.stderr: 2024-04-22T06:32:55.010 INFO:tasks.ceph.mds.a.smithi044.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T06:32:55.010 INFO:tasks.ceph.mds.a.smithi044.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f95f103e6f0] 2024-04-22T06:32:55.010 INFO:tasks.ceph.mds.a.smithi044.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f95f108b94c] 2024-04-22T06:32:55.010 INFO:tasks.ceph.mds.a.smithi044.stderr: 3: raise() 2024-04-22T06:32:55.010 INFO:tasks.ceph.mds.a.smithi044.stderr: 4: abort() 2024-04-22T06:32:55.010 INFO:tasks.ceph.mds.a.smithi044.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f95f1981fb6] 2024-04-22T06:32:55.010 INFO:tasks.ceph.mds.a.smithi044.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f95f198211a] 2024-04-22T06:32:55.010 INFO:tasks.ceph.mds.a.smithi044.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x5597e9caffd2] 2024-04-22T06:32:55.011 INFO:tasks.ceph.mds.a.smithi044.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x5597e9cb0091] 2024-04-22T06:32:55.011 INFO:tasks.ceph.mds.a.smithi044.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x5597e9c6aa96] 2024-04-22T06:32:55.011 INFO:tasks.ceph.mds.a.smithi044.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x5597e9c6a9f6] 2024-04-22T06:32:55.011 INFO:tasks.ceph.mds.a.smithi044.stderr: 11: ceph-mds(+0x48e3cf) [0x5597e9e083cf] 2024-04-22T06:32:55.011 INFO:tasks.ceph.mds.a.smithi044.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x5597e9e58c1b] 2024-04-22T06:32:55.011 INFO:tasks.ceph.mds.a.smithi044.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7f95f1089c02] 2024-04-22T06:32:55.011 INFO:tasks.ceph.mds.a.smithi044.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7f95f110ec40] 2024-04-22T06:32:55.011 INFO:tasks.ceph.mds.a.smithi044.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T06:32:55.011 INFO:tasks.ceph.mds.a.smithi044.stderr: 2024-04-22T06:32:55.011 INFO:tasks.ceph.mds.e.smithi044.stderr:2024-04-22T06:32:54.993+0000 7f5a98b76640 -1 *** Caught signal (Aborted) ** 2024-04-22T06:32:55.012 INFO:tasks.ceph.mds.e.smithi044.stderr: in thread 7f5a98b76640 thread_name:quiesce.agt 2024-04-22T06:32:55.012 INFO:tasks.ceph.mds.e.smithi044.stderr: 2024-04-22T06:32:55.012 INFO:tasks.ceph.mds.e.smithi044.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T06:32:55.012 INFO:tasks.ceph.mds.e.smithi044.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f5aa5e3e6f0] 2024-04-22T06:32:55.012 INFO:tasks.ceph.mds.e.smithi044.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f5aa5e8b94c] 2024-04-22T06:32:55.012 INFO:tasks.ceph.mds.e.smithi044.stderr: 3: raise() 2024-04-22T06:32:55.012 INFO:tasks.ceph.mds.e.smithi044.stderr: 4: abort() 2024-04-22T06:32:55.012 INFO:tasks.ceph.mds.e.smithi044.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f5aa6781fb6] 2024-04-22T06:32:55.012 INFO:tasks.ceph.mds.e.smithi044.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f5aa678211a] 2024-04-22T06:32:55.012 INFO:tasks.ceph.mds.e.smithi044.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x55ffb5ae9fd2] 2024-04-22T06:32:55.012 INFO:tasks.ceph.mds.e.smithi044.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x55ffb5aea091] 2024-04-22T06:32:55.012 INFO:tasks.ceph.mds.e.smithi044.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x55ffb5aa4a96] 2024-04-22T06:32:55.012 INFO:tasks.ceph.mds.e.smithi044.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x55ffb5aa49f6] 2024-04-22T06:32:55.012 INFO:tasks.ceph.mds.e.smithi044.stderr: 11: ceph-mds(+0x48e3cf) [0x55ffb5c423cf] 2024-04-22T06:32:55.012 INFO:tasks.ceph.mds.e.smithi044.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x55ffb5c92c1b] 2024-04-22T06:32:55.013 INFO:tasks.ceph.mds.e.smithi044.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7f5aa5e89c02] 2024-04-22T06:32:55.013 INFO:tasks.ceph.mds.e.smithi044.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7f5aa5f0ec40] 2024-04-22T06:32:55.013 INFO:tasks.ceph.mds.e.smithi044.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T06:32:55.013 INFO:tasks.ceph.mds.e.smithi044.stderr: 2024-04-22T06:32:55.013 INFO:tasks.ceph.mds.c.smithi044.stderr: -1> 2024-04-22T06:32:54.992+0000 7f7de9fa4640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7f7de9fa4640 time 2024-04-22T06:32:54.992905+0000 2024-04-22T06:32:55.013 INFO:tasks.ceph.mds.c.smithi044.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T06:32:55.013 INFO:tasks.ceph.mds.c.smithi044.stderr: 2024-04-22T06:32:55.013 INFO:tasks.ceph.mds.c.smithi044.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T06:32:55.013 INFO:tasks.ceph.mds.c.smithi044.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f7df7b81f5c] 2024-04-22T06:32:55.013 INFO:tasks.ceph.mds.c.smithi044.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f7df7b8211a] 2024-04-22T06:32:55.014 INFO:tasks.ceph.mds.c.smithi044.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x555a9dea7fd2] 2024-04-22T06:32:55.014 INFO:tasks.ceph.mds.c.smithi044.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x555a9dea8091] 2024-04-22T06:32:55.014 INFO:tasks.ceph.mds.c.smithi044.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x555a9de62a96] 2024-04-22T06:32:55.014 INFO:tasks.ceph.mds.c.smithi044.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x555a9de629f6] 2024-04-22T06:32:55.014 INFO:tasks.ceph.mds.c.smithi044.stderr: 7: ceph-mds(+0x48e3cf) [0x555a9e0003cf] 2024-04-22T06:32:55.014 INFO:tasks.ceph.mds.c.smithi044.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x555a9e050c1b] 2024-04-22T06:32:55.014 INFO:tasks.ceph.mds.c.smithi044.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7f7df7289c02] 2024-04-22T06:32:55.014 INFO:tasks.ceph.mds.c.smithi044.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7f7df730ec40] 2024-04-22T06:32:55.014 INFO:tasks.ceph.mds.c.smithi044.stderr: 2024-04-22T06:32:55.014 INFO:tasks.ceph.mds.c.smithi044.stderr: 0> 2024-04-22T06:32:54.993+0000 7f7de9fa4640 -1 *** Caught signal (Aborted) ** 2024-04-22T06:32:55.014 INFO:tasks.ceph.mds.c.smithi044.stderr: in thread 7f7de9fa4640 thread_name:quiesce.agt 2024-04-22T06:32:55.014 INFO:tasks.ceph.mds.c.smithi044.stderr: 2024-04-22T06:32:55.014 INFO:tasks.ceph.mds.c.smithi044.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T06:32:55.015 INFO:tasks.ceph.mds.c.smithi044.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f7df723e6f0] 2024-04-22T06:32:55.015 INFO:tasks.ceph.mds.c.smithi044.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f7df728b94c] 2024-04-22T06:32:55.015 INFO:tasks.ceph.mds.c.smithi044.stderr: 3: raise() 2024-04-22T06:32:55.015 INFO:tasks.ceph.mds.c.smithi044.stderr: 4: abort() 2024-04-22T06:32:55.015 INFO:tasks.ceph.mds.c.smithi044.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f7df7b81fb6] 2024-04-22T06:32:55.015 INFO:tasks.ceph.mds.c.smithi044.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f7df7b8211a] 2024-04-22T06:32:55.015 INFO:tasks.ceph.mds.c.smithi044.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x555a9dea7fd2] 2024-04-22T06:32:55.015 INFO:tasks.ceph.mds.c.smithi044.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x555a9dea8091] 2024-04-22T06:32:55.015 INFO:tasks.ceph.mds.c.smithi044.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x555a9de62a96] 2024-04-22T06:32:55.015 INFO:tasks.ceph.mds.c.smithi044.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x555a9de629f6] 2024-04-22T06:32:55.015 INFO:tasks.ceph.mds.c.smithi044.stderr: 11: ceph-mds(+0x48e3cf) [0x555a9e0003cf] 2024-04-22T06:32:55.015 INFO:tasks.ceph.mds.c.smithi044.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x555a9e050c1b] 2024-04-22T06:32:55.015 INFO:tasks.ceph.mds.c.smithi044.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7f7df7289c02] 2024-04-22T06:32:55.015 INFO:tasks.ceph.mds.c.smithi044.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7f7df730ec40] 2024-04-22T06:32:55.016 INFO:tasks.ceph.mds.c.smithi044.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T06:32:55.016 INFO:tasks.ceph.mds.c.smithi044.stderr: 2024-04-22T06:32:55.016 INFO:tasks.ceph.mds.e.smithi044.stderr: -1> 2024-04-22T06:32:54.992+0000 7f5a98b76640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7f5a98b76640 time 2024-04-22T06:32:54.992922+0000 2024-04-22T06:32:55.016 INFO:tasks.ceph.mds.e.smithi044.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T06:32:55.016 INFO:tasks.ceph.mds.e.smithi044.stderr: 2024-04-22T06:32:55.016 INFO:tasks.ceph.mds.e.smithi044.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T06:32:55.016 INFO:tasks.ceph.mds.e.smithi044.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f5aa6781f5c] 2024-04-22T06:32:55.016 INFO:tasks.ceph.mds.e.smithi044.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f5aa678211a] 2024-04-22T06:32:55.016 INFO:tasks.ceph.mds.e.smithi044.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x55ffb5ae9fd2] 2024-04-22T06:32:55.016 INFO:tasks.ceph.mds.e.smithi044.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x55ffb5aea091] 2024-04-22T06:32:55.016 INFO:tasks.ceph.mds.e.smithi044.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x55ffb5aa4a96] 2024-04-22T06:32:55.017 INFO:tasks.ceph.mds.e.smithi044.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x55ffb5aa49f6] 2024-04-22T06:32:55.017 INFO:tasks.ceph.mds.e.smithi044.stderr: 7: ceph-mds(+0x48e3cf) [0x55ffb5c423cf] 2024-04-22T06:32:55.017 INFO:tasks.ceph.mds.e.smithi044.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x55ffb5c92c1b] 2024-04-22T06:32:55.017 INFO:tasks.ceph.mds.e.smithi044.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7f5aa5e89c02] 2024-04-22T06:32:55.017 INFO:tasks.ceph.mds.e.smithi044.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7f5aa5f0ec40] 2024-04-22T06:32:55.017 INFO:tasks.ceph.mds.e.smithi044.stderr: 2024-04-22T06:32:55.017 INFO:tasks.ceph.mds.e.smithi044.stderr: 0> 2024-04-22T06:32:54.993+0000 7f5a98b76640 -1 *** Caught signal (Aborted) ** 2024-04-22T06:32:55.017 INFO:tasks.ceph.mds.e.smithi044.stderr: in thread 7f5a98b76640 thread_name:quiesce.agt 2024-04-22T06:32:55.017 INFO:tasks.ceph.mds.e.smithi044.stderr: 2024-04-22T06:32:55.017 INFO:tasks.ceph.mds.e.smithi044.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T06:32:55.017 INFO:tasks.ceph.mds.e.smithi044.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f5aa5e3e6f0] 2024-04-22T06:32:55.017 INFO:tasks.ceph.mds.e.smithi044.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f5aa5e8b94c] 2024-04-22T06:32:55.017 INFO:tasks.ceph.mds.e.smithi044.stderr: 3: raise() 2024-04-22T06:32:55.017 INFO:tasks.ceph.mds.e.smithi044.stderr: 4: abort() 2024-04-22T06:32:55.017 INFO:tasks.ceph.mds.e.smithi044.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f5aa6781fb6] 2024-04-22T06:32:55.017 INFO:tasks.ceph.mds.e.smithi044.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f5aa678211a] 2024-04-22T06:32:55.018 INFO:tasks.ceph.mds.e.smithi044.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x55ffb5ae9fd2] 2024-04-22T06:32:55.018 INFO:tasks.ceph.mds.e.smithi044.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x55ffb5aea091] 2024-04-22T06:32:55.018 INFO:tasks.ceph.mds.e.smithi044.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x55ffb5aa4a96] 2024-04-22T06:32:55.018 INFO:tasks.ceph.mds.e.smithi044.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x55ffb5aa49f6] 2024-04-22T06:32:55.018 INFO:tasks.ceph.mds.e.smithi044.stderr: 11: ceph-mds(+0x48e3cf) [0x55ffb5c423cf] 2024-04-22T06:32:55.018 INFO:tasks.ceph.mds.e.smithi044.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x55ffb5c92c1b] 2024-04-22T06:32:55.018 INFO:tasks.ceph.mds.e.smithi044.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7f5aa5e89c02] 2024-04-22T06:32:55.018 INFO:tasks.ceph.mds.e.smithi044.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7f5aa5f0ec40] 2024-04-22T06:32:55.018 INFO:tasks.ceph.mds.e.smithi044.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T06:32:55.018 INFO:tasks.ceph.mds.e.smithi044.stderr: 2024-04-22T06:32:55.019 INFO:tasks.ceph.mds.a.smithi044.stderr: -1> 2024-04-22T06:32:54.992+0000 7f95e3e18640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7f95e3e18640 time 2024-04-22T06:32:54.992849+0000 2024-04-22T06:32:55.019 INFO:tasks.ceph.mds.a.smithi044.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T06:32:55.019 INFO:tasks.ceph.mds.a.smithi044.stderr: 2024-04-22T06:32:55.019 INFO:tasks.ceph.mds.a.smithi044.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T06:32:55.019 INFO:tasks.ceph.mds.a.smithi044.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f95f1981f5c] 2024-04-22T06:32:55.019 INFO:tasks.ceph.mds.a.smithi044.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f95f198211a] 2024-04-22T06:32:55.019 INFO:tasks.ceph.mds.a.smithi044.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x5597e9caffd2] 2024-04-22T06:32:55.019 INFO:tasks.ceph.mds.a.smithi044.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x5597e9cb0091] 2024-04-22T06:32:55.020 INFO:tasks.ceph.mds.a.smithi044.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x5597e9c6aa96] 2024-04-22T06:32:55.020 INFO:tasks.ceph.mds.a.smithi044.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x5597e9c6a9f6] 2024-04-22T06:32:55.020 INFO:tasks.ceph.mds.a.smithi044.stderr: 7: ceph-mds(+0x48e3cf) [0x5597e9e083cf] 2024-04-22T06:32:55.020 INFO:tasks.ceph.mds.a.smithi044.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x5597e9e58c1b] 2024-04-22T06:32:55.020 INFO:tasks.ceph.mds.a.smithi044.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7f95f1089c02] 2024-04-22T06:32:55.020 INFO:tasks.ceph.mds.a.smithi044.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7f95f110ec40] 2024-04-22T06:32:55.020 INFO:tasks.ceph.mds.a.smithi044.stderr: 2024-04-22T06:32:55.020 INFO:tasks.ceph.mds.a.smithi044.stderr: 0> 2024-04-22T06:32:54.993+0000 7f95e3e18640 -1 *** Caught signal (Aborted) ** 2024-04-22T06:32:55.020 INFO:tasks.ceph.mds.a.smithi044.stderr: in thread 7f95e3e18640 thread_name:quiesce.agt 2024-04-22T06:32:55.020 INFO:tasks.ceph.mds.a.smithi044.stderr: 2024-04-22T06:32:55.020 INFO:tasks.ceph.mds.a.smithi044.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T06:32:55.020 INFO:tasks.ceph.mds.a.smithi044.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f95f103e6f0] 2024-04-22T06:32:55.020 INFO:tasks.ceph.mds.a.smithi044.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f95f108b94c] 2024-04-22T06:32:55.020 INFO:tasks.ceph.mds.a.smithi044.stderr: 3: raise() 2024-04-22T06:32:55.021 INFO:tasks.ceph.mds.a.smithi044.stderr: 4: abort() 2024-04-22T06:32:55.021 INFO:tasks.ceph.mds.a.smithi044.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f95f1981fb6] 2024-04-22T06:32:55.021 INFO:tasks.ceph.mds.a.smithi044.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f95f198211a] 2024-04-22T06:32:55.021 INFO:tasks.ceph.mds.a.smithi044.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x5597e9caffd2] 2024-04-22T06:32:55.021 INFO:tasks.ceph.mds.a.smithi044.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x5597e9cb0091] 2024-04-22T06:32:55.021 INFO:tasks.ceph.mds.a.smithi044.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x5597e9c6aa96] 2024-04-22T06:32:55.021 INFO:tasks.ceph.mds.a.smithi044.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x5597e9c6a9f6] 2024-04-22T06:32:55.021 INFO:tasks.ceph.mds.a.smithi044.stderr: 11: ceph-mds(+0x48e3cf) [0x5597e9e083cf] 2024-04-22T06:32:55.021 INFO:tasks.ceph.mds.a.smithi044.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x5597e9e58c1b] 2024-04-22T06:32:55.021 INFO:tasks.ceph.mds.a.smithi044.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7f95f1089c02] 2024-04-22T06:32:55.021 INFO:tasks.ceph.mds.a.smithi044.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7f95f110ec40] 2024-04-22T06:32:55.021 INFO:tasks.ceph.mds.a.smithi044.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T06:32:55.021 INFO:tasks.ceph.mds.a.smithi044.stderr: 2024-04-22T06:32:55.022 INFO:tasks.ceph.mds.c.smithi044.stderr: -1> 2024-04-22T06:32:54.992+0000 7f7de9fa4640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7f7de9fa4640 time 2024-04-22T06:32:54.992905+0000 2024-04-22T06:32:55.022 INFO:tasks.ceph.mds.c.smithi044.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T06:32:55.022 INFO:tasks.ceph.mds.c.smithi044.stderr: 2024-04-22T06:32:55.022 INFO:tasks.ceph.mds.c.smithi044.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T06:32:55.022 INFO:tasks.ceph.mds.c.smithi044.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f7df7b81f5c] 2024-04-22T06:32:55.022 INFO:tasks.ceph.mds.c.smithi044.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f7df7b8211a] 2024-04-22T06:32:55.022 INFO:tasks.ceph.mds.c.smithi044.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x555a9dea7fd2] 2024-04-22T06:32:55.022 INFO:tasks.ceph.mds.c.smithi044.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x555a9dea8091] 2024-04-22T06:32:55.022 INFO:tasks.ceph.mds.c.smithi044.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x555a9de62a96] 2024-04-22T06:32:55.023 INFO:tasks.ceph.mds.c.smithi044.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x555a9de629f6] 2024-04-22T06:32:55.023 INFO:tasks.ceph.mds.c.smithi044.stderr: 7: ceph-mds(+0x48e3cf) [0x555a9e0003cf] 2024-04-22T06:32:55.023 INFO:tasks.ceph.mds.c.smithi044.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x555a9e050c1b] 2024-04-22T06:32:55.023 INFO:tasks.ceph.mds.c.smithi044.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7f7df7289c02] 2024-04-22T06:32:55.023 INFO:tasks.ceph.mds.c.smithi044.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7f7df730ec40] 2024-04-22T06:32:55.023 INFO:tasks.ceph.mds.c.smithi044.stderr: 2024-04-22T06:32:55.023 INFO:tasks.ceph.mds.c.smithi044.stderr: 0> 2024-04-22T06:32:54.993+0000 7f7de9fa4640 -1 *** Caught signal (Aborted) ** 2024-04-22T06:32:55.023 INFO:tasks.ceph.mds.c.smithi044.stderr: in thread 7f7de9fa4640 thread_name:quiesce.agt 2024-04-22T06:32:55.023 INFO:tasks.ceph.mds.c.smithi044.stderr: 2024-04-22T06:32:55.023 INFO:tasks.ceph.mds.c.smithi044.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T06:32:55.023 INFO:tasks.ceph.mds.c.smithi044.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f7df723e6f0] 2024-04-22T06:32:55.023 INFO:tasks.ceph.mds.c.smithi044.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f7df728b94c] 2024-04-22T06:32:55.023 INFO:tasks.ceph.mds.c.smithi044.stderr: 3: raise() 2024-04-22T06:32:55.023 INFO:tasks.ceph.mds.c.smithi044.stderr: 4: abort() 2024-04-22T06:32:55.023 INFO:tasks.ceph.mds.c.smithi044.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f7df7b81fb6] 2024-04-22T06:32:55.023 INFO:tasks.ceph.mds.c.smithi044.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f7df7b8211a] 2024-04-22T06:32:55.024 INFO:tasks.ceph.mds.c.smithi044.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x555a9dea7fd2] 2024-04-22T06:32:55.024 INFO:tasks.ceph.mds.c.smithi044.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x555a9dea8091] 2024-04-22T06:32:55.024 INFO:tasks.ceph.mds.c.smithi044.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x555a9de62a96] 2024-04-22T06:32:55.024 INFO:tasks.ceph.mds.c.smithi044.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x555a9de629f6] 2024-04-22T06:32:55.024 INFO:tasks.ceph.mds.c.smithi044.stderr: 11: ceph-mds(+0x48e3cf) [0x555a9e0003cf] 2024-04-22T06:32:55.024 INFO:tasks.ceph.mds.c.smithi044.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x555a9e050c1b] 2024-04-22T06:32:55.024 INFO:tasks.ceph.mds.c.smithi044.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7f7df7289c02] 2024-04-22T06:32:55.024 INFO:tasks.ceph.mds.c.smithi044.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7f7df730ec40] 2024-04-22T06:32:55.024 INFO:tasks.ceph.mds.c.smithi044.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T06:32:55.024 INFO:tasks.ceph.mds.c.smithi044.stderr: 2024-04-22T06:32:55.025 INFO:tasks.ceph.mds.a.smithi044.stderr: -1> 2024-04-22T06:32:54.992+0000 7f95e3e18640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7f95e3e18640 time 2024-04-22T06:32:54.992849+0000 2024-04-22T06:32:55.025 INFO:tasks.ceph.mds.a.smithi044.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T06:32:55.025 INFO:tasks.ceph.mds.a.smithi044.stderr: 2024-04-22T06:32:55.025 INFO:tasks.ceph.mds.a.smithi044.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T06:32:55.025 INFO:tasks.ceph.mds.a.smithi044.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f95f1981f5c] 2024-04-22T06:32:55.025 INFO:tasks.ceph.mds.a.smithi044.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f95f198211a] 2024-04-22T06:32:55.025 INFO:tasks.ceph.mds.a.smithi044.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x5597e9caffd2] 2024-04-22T06:32:55.025 INFO:tasks.ceph.mds.a.smithi044.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x5597e9cb0091] 2024-04-22T06:32:55.025 INFO:tasks.ceph.mds.a.smithi044.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x5597e9c6aa96] 2024-04-22T06:32:55.025 INFO:tasks.ceph.mds.a.smithi044.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x5597e9c6a9f6] 2024-04-22T06:32:55.025 INFO:tasks.ceph.mds.a.smithi044.stderr: 7: ceph-mds(+0x48e3cf) [0x5597e9e083cf] 2024-04-22T06:32:55.026 INFO:tasks.ceph.mds.a.smithi044.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x5597e9e58c1b] 2024-04-22T06:32:55.026 INFO:tasks.ceph.mds.a.smithi044.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7f95f1089c02] 2024-04-22T06:32:55.026 INFO:tasks.ceph.mds.a.smithi044.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7f95f110ec40] 2024-04-22T06:32:55.026 INFO:tasks.ceph.mds.a.smithi044.stderr: 2024-04-22T06:32:55.026 INFO:tasks.ceph.mds.a.smithi044.stderr: 0> 2024-04-22T06:32:54.993+0000 7f95e3e18640 -1 *** Caught signal (Aborted) ** 2024-04-22T06:32:55.026 INFO:tasks.ceph.mds.a.smithi044.stderr: in thread 7f95e3e18640 thread_name:quiesce.agt 2024-04-22T06:32:55.026 INFO:tasks.ceph.mds.a.smithi044.stderr: 2024-04-22T06:32:55.026 INFO:tasks.ceph.mds.a.smithi044.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T06:32:55.026 INFO:tasks.ceph.mds.a.smithi044.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f95f103e6f0] 2024-04-22T06:32:55.026 INFO:tasks.ceph.mds.a.smithi044.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f95f108b94c] 2024-04-22T06:32:55.026 INFO:tasks.ceph.mds.a.smithi044.stderr: 3: raise() 2024-04-22T06:32:55.026 INFO:tasks.ceph.mds.a.smithi044.stderr: 4: abort() 2024-04-22T06:32:55.026 INFO:tasks.ceph.mds.a.smithi044.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f95f1981fb6] 2024-04-22T06:32:55.026 INFO:tasks.ceph.mds.a.smithi044.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f95f198211a] 2024-04-22T06:32:55.026 INFO:tasks.ceph.mds.a.smithi044.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x5597e9caffd2] 2024-04-22T06:32:55.027 INFO:tasks.ceph.mds.a.smithi044.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x5597e9cb0091] 2024-04-22T06:32:55.027 INFO:tasks.ceph.mds.a.smithi044.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x5597e9c6aa96] 2024-04-22T06:32:55.027 INFO:tasks.ceph.mds.a.smithi044.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x5597e9c6a9f6] 2024-04-22T06:32:55.027 INFO:tasks.ceph.mds.a.smithi044.stderr: 11: ceph-mds(+0x48e3cf) [0x5597e9e083cf] 2024-04-22T06:32:55.027 INFO:tasks.ceph.mds.a.smithi044.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x5597e9e58c1b] 2024-04-22T06:32:55.027 INFO:tasks.ceph.mds.a.smithi044.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7f95f1089c02] 2024-04-22T06:32:55.027 INFO:tasks.ceph.mds.a.smithi044.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7f95f110ec40] 2024-04-22T06:32:55.027 INFO:tasks.ceph.mds.a.smithi044.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T06:32:55.027 INFO:tasks.ceph.mds.a.smithi044.stderr: 2024-04-22T06:32:55.027 INFO:tasks.ceph.mds.e.smithi044.stderr: -1> 2024-04-22T06:32:54.992+0000 7f5a98b76640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7f5a98b76640 time 2024-04-22T06:32:54.992922+0000 2024-04-22T06:32:55.027 INFO:tasks.ceph.mds.e.smithi044.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T06:32:55.028 INFO:tasks.ceph.mds.e.smithi044.stderr: 2024-04-22T06:32:55.028 INFO:tasks.ceph.mds.e.smithi044.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T06:32:55.028 INFO:tasks.ceph.mds.e.smithi044.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f5aa6781f5c] 2024-04-22T06:32:55.028 INFO:tasks.ceph.mds.e.smithi044.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f5aa678211a] 2024-04-22T06:32:55.028 INFO:tasks.ceph.mds.e.smithi044.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x55ffb5ae9fd2] 2024-04-22T06:32:55.028 INFO:tasks.ceph.mds.e.smithi044.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x55ffb5aea091] 2024-04-22T06:32:55.028 INFO:tasks.ceph.mds.e.smithi044.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x55ffb5aa4a96] 2024-04-22T06:32:55.028 INFO:tasks.ceph.mds.e.smithi044.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x55ffb5aa49f6] 2024-04-22T06:32:55.028 INFO:tasks.ceph.mds.e.smithi044.stderr: 7: ceph-mds(+0x48e3cf) [0x55ffb5c423cf] 2024-04-22T06:32:55.028 INFO:tasks.ceph.mds.e.smithi044.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x55ffb5c92c1b] 2024-04-22T06:32:55.028 INFO:tasks.ceph.mds.e.smithi044.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7f5aa5e89c02] 2024-04-22T06:32:55.028 INFO:tasks.ceph.mds.e.smithi044.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7f5aa5f0ec40] 2024-04-22T06:32:55.028 INFO:tasks.ceph.mds.e.smithi044.stderr: 2024-04-22T06:32:55.028 INFO:tasks.ceph.mds.e.smithi044.stderr: 0> 2024-04-22T06:32:54.993+0000 7f5a98b76640 -1 *** Caught signal (Aborted) ** 2024-04-22T06:32:55.029 INFO:tasks.ceph.mds.e.smithi044.stderr: in thread 7f5a98b76640 thread_name:quiesce.agt 2024-04-22T06:32:55.029 INFO:tasks.ceph.mds.e.smithi044.stderr: 2024-04-22T06:32:55.029 INFO:tasks.ceph.mds.e.smithi044.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T06:32:55.029 INFO:tasks.ceph.mds.e.smithi044.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f5aa5e3e6f0] 2024-04-22T06:32:55.029 INFO:tasks.ceph.mds.e.smithi044.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f5aa5e8b94c] 2024-04-22T06:32:55.029 INFO:tasks.ceph.mds.e.smithi044.stderr: 3: raise() 2024-04-22T06:32:55.029 INFO:tasks.ceph.mds.e.smithi044.stderr: 4: abort() 2024-04-22T06:32:55.029 INFO:tasks.ceph.mds.e.smithi044.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f5aa6781fb6] 2024-04-22T06:32:55.029 INFO:tasks.ceph.mds.e.smithi044.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f5aa678211a] 2024-04-22T06:32:55.029 INFO:tasks.ceph.mds.e.smithi044.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x55ffb5ae9fd2] 2024-04-22T06:32:55.029 INFO:tasks.ceph.mds.e.smithi044.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x55ffb5aea091] 2024-04-22T06:32:55.029 INFO:tasks.ceph.mds.e.smithi044.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x55ffb5aa4a96] 2024-04-22T06:32:55.029 INFO:tasks.ceph.mds.e.smithi044.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x55ffb5aa49f6] 2024-04-22T06:32:55.029 INFO:tasks.ceph.mds.e.smithi044.stderr: 11: ceph-mds(+0x48e3cf) [0x55ffb5c423cf] 2024-04-22T06:32:55.029 INFO:tasks.ceph.mds.e.smithi044.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x55ffb5c92c1b] 2024-04-22T06:32:55.029 INFO:tasks.ceph.mds.e.smithi044.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7f5aa5e89c02] 2024-04-22T06:32:55.030 INFO:tasks.ceph.mds.e.smithi044.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7f5aa5f0ec40] 2024-04-22T06:32:55.030 INFO:tasks.ceph.mds.e.smithi044.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T06:32:55.030 INFO:tasks.ceph.mds.e.smithi044.stderr: 2024-04-22T06:32:55.030 INFO:tasks.ceph.mds.b.smithi070.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7f8389b93640 time 2024-04-22T06:32:54.992917+0000 2024-04-22T06:32:55.030 INFO:tasks.ceph.mds.b.smithi070.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T06:32:55.030 INFO:tasks.ceph.mds.b.smithi070.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T06:32:55.031 INFO:tasks.ceph.mds.b.smithi070.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f8397781f5c] 2024-04-22T06:32:55.031 INFO:tasks.ceph.mds.b.smithi070.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f839778211a] 2024-04-22T06:32:55.031 INFO:tasks.ceph.mds.b.smithi070.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x55faf4808fd2] 2024-04-22T06:32:55.031 INFO:tasks.ceph.mds.b.smithi070.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x55faf4809091] 2024-04-22T06:32:55.031 INFO:tasks.ceph.mds.b.smithi070.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x55faf47c3a96] 2024-04-22T06:32:55.031 INFO:tasks.ceph.mds.b.smithi070.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x55faf47c39f6] 2024-04-22T06:32:55.031 INFO:tasks.ceph.mds.b.smithi070.stderr: 7: ceph-mds(+0x48e3cf) [0x55faf49613cf] 2024-04-22T06:32:55.031 INFO:tasks.ceph.mds.b.smithi070.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x55faf49b1c1b] 2024-04-22T06:32:55.031 INFO:tasks.ceph.mds.b.smithi070.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7f8396e89c02] 2024-04-22T06:32:55.031 INFO:tasks.ceph.mds.b.smithi070.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7f8396f0ec40] 2024-04-22T06:32:55.031 INFO:tasks.ceph.mds.b.smithi070.stderr:2024-04-22T06:32:54.992+0000 7f8389b93640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7f8389b93640 time 2024-04-22T06:32:54.992917+0000 2024-04-22T06:32:55.031 INFO:tasks.ceph.mds.b.smithi070.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T06:32:55.031 INFO:tasks.ceph.mds.b.smithi070.stderr: 2024-04-22T06:32:55.031 INFO:tasks.ceph.mds.b.smithi070.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T06:32:55.032 INFO:tasks.ceph.mds.b.smithi070.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f8397781f5c] 2024-04-22T06:32:55.032 INFO:tasks.ceph.mds.b.smithi070.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f839778211a] 2024-04-22T06:32:55.032 INFO:tasks.ceph.mds.b.smithi070.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x55faf4808fd2] 2024-04-22T06:32:55.032 INFO:tasks.ceph.mds.b.smithi070.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x55faf4809091] 2024-04-22T06:32:55.032 INFO:tasks.ceph.mds.b.smithi070.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x55faf47c3a96] 2024-04-22T06:32:55.032 INFO:tasks.ceph.mds.b.smithi070.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x55faf47c39f6] 2024-04-22T06:32:55.032 INFO:tasks.ceph.mds.b.smithi070.stderr: 7: ceph-mds(+0x48e3cf) [0x55faf49613cf] 2024-04-22T06:32:55.032 INFO:tasks.ceph.mds.b.smithi070.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x55faf49b1c1b] 2024-04-22T06:32:55.032 INFO:tasks.ceph.mds.b.smithi070.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7f8396e89c02] 2024-04-22T06:32:55.032 INFO:tasks.ceph.mds.b.smithi070.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7f8396f0ec40] 2024-04-22T06:32:55.032 INFO:tasks.ceph.mds.b.smithi070.stderr: 2024-04-22T06:32:55.032 INFO:tasks.ceph.mds.b.smithi070.stderr:*** Caught signal (Aborted) ** 2024-04-22T06:32:55.032 INFO:tasks.ceph.mds.b.smithi070.stderr: in thread 7f8389b93640 thread_name:quiesce.agt 2024-04-22T06:32:55.033 INFO:tasks.ceph.mds.b.smithi070.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T06:32:55.033 INFO:tasks.ceph.mds.b.smithi070.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f8396e3e6f0] 2024-04-22T06:32:55.033 INFO:tasks.ceph.mds.b.smithi070.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f8396e8b94c] 2024-04-22T06:32:55.033 INFO:tasks.ceph.mds.b.smithi070.stderr: 3: raise() 2024-04-22T06:32:55.033 INFO:tasks.ceph.mds.b.smithi070.stderr: 4: abort() 2024-04-22T06:32:55.033 INFO:tasks.ceph.mds.b.smithi070.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f8397781fb6] 2024-04-22T06:32:55.033 INFO:tasks.ceph.mds.b.smithi070.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f839778211a] 2024-04-22T06:32:55.033 INFO:tasks.ceph.mds.b.smithi070.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x55faf4808fd2] 2024-04-22T06:32:55.033 INFO:tasks.ceph.mds.b.smithi070.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x55faf4809091] 2024-04-22T06:32:55.033 INFO:tasks.ceph.mds.b.smithi070.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x55faf47c3a96] 2024-04-22T06:32:55.033 INFO:tasks.ceph.mds.b.smithi070.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x55faf47c39f6] 2024-04-22T06:32:55.033 INFO:tasks.ceph.mds.b.smithi070.stderr: 11: ceph-mds(+0x48e3cf) [0x55faf49613cf] 2024-04-22T06:32:55.033 INFO:tasks.ceph.mds.b.smithi070.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x55faf49b1c1b] 2024-04-22T06:32:55.033 INFO:tasks.ceph.mds.b.smithi070.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7f8396e89c02] 2024-04-22T06:32:55.034 INFO:tasks.ceph.mds.b.smithi070.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7f8396f0ec40] 2024-04-22T06:32:55.034 INFO:tasks.ceph.mds.b.smithi070.stderr:2024-04-22T06:32:54.992+0000 7f8389b93640 -1 *** Caught signal (Aborted) ** 2024-04-22T06:32:55.034 INFO:tasks.ceph.mds.b.smithi070.stderr: in thread 7f8389b93640 thread_name:quiesce.agt 2024-04-22T06:32:55.034 INFO:tasks.ceph.mds.b.smithi070.stderr: 2024-04-22T06:32:55.034 INFO:tasks.ceph.mds.b.smithi070.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T06:32:55.034 INFO:tasks.ceph.mds.b.smithi070.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f8396e3e6f0] 2024-04-22T06:32:55.034 INFO:tasks.ceph.mds.b.smithi070.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f8396e8b94c] 2024-04-22T06:32:55.034 INFO:tasks.ceph.mds.b.smithi070.stderr: 3: raise() 2024-04-22T06:32:55.034 INFO:tasks.ceph.mds.b.smithi070.stderr: 4: abort() 2024-04-22T06:32:55.034 INFO:tasks.ceph.mds.b.smithi070.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f8397781fb6] 2024-04-22T06:32:55.034 INFO:tasks.ceph.mds.b.smithi070.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f839778211a] 2024-04-22T06:32:55.034 INFO:tasks.ceph.mds.b.smithi070.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x55faf4808fd2] 2024-04-22T06:32:55.034 INFO:tasks.ceph.mds.b.smithi070.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x55faf4809091] 2024-04-22T06:32:55.034 INFO:tasks.ceph.mds.b.smithi070.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x55faf47c3a96] 2024-04-22T06:32:55.035 INFO:tasks.ceph.mds.b.smithi070.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x55faf47c39f6] 2024-04-22T06:32:55.035 INFO:tasks.ceph.mds.b.smithi070.stderr: 11: ceph-mds(+0x48e3cf) [0x55faf49613cf] 2024-04-22T06:32:55.035 INFO:tasks.ceph.mds.b.smithi070.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x55faf49b1c1b] 2024-04-22T06:32:55.035 INFO:tasks.ceph.mds.b.smithi070.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7f8396e89c02] 2024-04-22T06:32:55.035 INFO:tasks.ceph.mds.b.smithi070.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7f8396f0ec40] 2024-04-22T06:32:55.035 INFO:tasks.ceph.mds.b.smithi070.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T06:32:55.035 INFO:tasks.ceph.mds.b.smithi070.stderr: 2024-04-22T06:32:55.035 INFO:tasks.ceph.mds.b.smithi070.stderr: -3755> 2024-04-22T06:31:00.105+0000 7f83953aa640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6837/0: (98) Address already in use 2024-04-22T06:32:55.035 INFO:tasks.ceph.mds.b.smithi070.stderr: -1> 2024-04-22T06:32:54.992+0000 7f8389b93640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7f8389b93640 time 2024-04-22T06:32:54.992917+0000 2024-04-22T06:32:55.035 INFO:tasks.ceph.mds.b.smithi070.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T06:32:55.035 INFO:tasks.ceph.mds.b.smithi070.stderr: 2024-04-22T06:32:55.035 INFO:tasks.ceph.mds.b.smithi070.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T06:32:55.035 INFO:tasks.ceph.mds.b.smithi070.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f8397781f5c] 2024-04-22T06:32:55.036 INFO:tasks.ceph.mds.b.smithi070.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f839778211a] 2024-04-22T06:32:55.036 INFO:tasks.ceph.mds.b.smithi070.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x55faf4808fd2] 2024-04-22T06:32:55.036 INFO:tasks.ceph.mds.b.smithi070.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x55faf4809091] 2024-04-22T06:32:55.036 INFO:tasks.ceph.mds.b.smithi070.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x55faf47c3a96] 2024-04-22T06:32:55.036 INFO:tasks.ceph.mds.b.smithi070.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x55faf47c39f6] 2024-04-22T06:32:55.036 INFO:tasks.ceph.mds.b.smithi070.stderr: 7: ceph-mds(+0x48e3cf) [0x55faf49613cf] 2024-04-22T06:32:55.036 INFO:tasks.ceph.mds.b.smithi070.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x55faf49b1c1b] 2024-04-22T06:32:55.036 INFO:tasks.ceph.mds.b.smithi070.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7f8396e89c02] 2024-04-22T06:32:55.036 INFO:tasks.ceph.mds.b.smithi070.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7f8396f0ec40] 2024-04-22T06:32:55.036 INFO:tasks.ceph.mds.b.smithi070.stderr: 2024-04-22T06:32:55.036 INFO:tasks.ceph.mds.b.smithi070.stderr: 0> 2024-04-22T06:32:54.992+0000 7f8389b93640 -1 *** Caught signal (Aborted) ** 2024-04-22T06:32:55.036 INFO:tasks.ceph.mds.b.smithi070.stderr: in thread 7f8389b93640 thread_name:quiesce.agt 2024-04-22T06:32:55.036 INFO:tasks.ceph.mds.b.smithi070.stderr: 2024-04-22T06:32:55.036 INFO:tasks.ceph.mds.b.smithi070.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T06:32:55.036 INFO:tasks.ceph.mds.b.smithi070.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f8396e3e6f0] 2024-04-22T06:32:55.037 INFO:tasks.ceph.mds.b.smithi070.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f8396e8b94c] 2024-04-22T06:32:55.037 INFO:tasks.ceph.mds.b.smithi070.stderr: 3: raise() 2024-04-22T06:32:55.037 INFO:tasks.ceph.mds.b.smithi070.stderr: 4: abort() 2024-04-22T06:32:55.037 INFO:tasks.ceph.mds.b.smithi070.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f8397781fb6] 2024-04-22T06:32:55.037 INFO:tasks.ceph.mds.b.smithi070.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f839778211a] 2024-04-22T06:32:55.037 INFO:tasks.ceph.mds.b.smithi070.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x55faf4808fd2] 2024-04-22T06:32:55.037 INFO:tasks.ceph.mds.b.smithi070.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x55faf4809091] 2024-04-22T06:32:55.037 INFO:tasks.ceph.mds.b.smithi070.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x55faf47c3a96] 2024-04-22T06:32:55.037 INFO:tasks.ceph.mds.b.smithi070.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x55faf47c39f6] 2024-04-22T06:32:55.037 INFO:tasks.ceph.mds.b.smithi070.stderr: 11: ceph-mds(+0x48e3cf) [0x55faf49613cf] 2024-04-22T06:32:55.037 INFO:tasks.ceph.mds.b.smithi070.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x55faf49b1c1b] 2024-04-22T06:32:55.037 INFO:tasks.ceph.mds.b.smithi070.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7f8396e89c02] 2024-04-22T06:32:55.037 INFO:tasks.ceph.mds.b.smithi070.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7f8396f0ec40] 2024-04-22T06:32:55.037 INFO:tasks.ceph.mds.b.smithi070.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T06:32:55.038 INFO:tasks.ceph.mds.b.smithi070.stderr: 2024-04-22T06:32:55.038 INFO:tasks.ceph.mds.b.smithi070.stderr: -3755> 2024-04-22T06:31:00.105+0000 7f83953aa640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6837/0: (98) Address already in use 2024-04-22T06:32:55.038 INFO:tasks.ceph.mds.b.smithi070.stderr: -1> 2024-04-22T06:32:54.992+0000 7f8389b93640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7f8389b93640 time 2024-04-22T06:32:54.992917+0000 2024-04-22T06:32:55.038 INFO:tasks.ceph.mds.b.smithi070.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T06:32:55.038 INFO:tasks.ceph.mds.b.smithi070.stderr: 2024-04-22T06:32:55.038 INFO:tasks.ceph.mds.b.smithi070.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T06:32:55.038 INFO:tasks.ceph.mds.b.smithi070.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f8397781f5c] 2024-04-22T06:32:55.038 INFO:tasks.ceph.mds.b.smithi070.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f839778211a] 2024-04-22T06:32:55.038 INFO:tasks.ceph.mds.b.smithi070.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x55faf4808fd2] 2024-04-22T06:32:55.038 INFO:tasks.ceph.mds.b.smithi070.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x55faf4809091] 2024-04-22T06:32:55.038 INFO:tasks.ceph.mds.b.smithi070.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x55faf47c3a96] 2024-04-22T06:32:55.038 INFO:tasks.ceph.mds.b.smithi070.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x55faf47c39f6] 2024-04-22T06:32:55.038 INFO:tasks.ceph.mds.b.smithi070.stderr: 7: ceph-mds(+0x48e3cf) [0x55faf49613cf] 2024-04-22T06:32:55.039 INFO:tasks.ceph.mds.b.smithi070.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x55faf49b1c1b] 2024-04-22T06:32:55.039 INFO:tasks.ceph.mds.b.smithi070.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7f8396e89c02] 2024-04-22T06:32:55.039 INFO:tasks.ceph.mds.b.smithi070.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7f8396f0ec40] 2024-04-22T06:32:55.039 INFO:tasks.ceph.mds.b.smithi070.stderr: 2024-04-22T06:32:55.039 INFO:tasks.ceph.mds.b.smithi070.stderr: 0> 2024-04-22T06:32:54.992+0000 7f8389b93640 -1 *** Caught signal (Aborted) ** 2024-04-22T06:32:55.039 INFO:tasks.ceph.mds.b.smithi070.stderr: in thread 7f8389b93640 thread_name:quiesce.agt 2024-04-22T06:32:55.039 INFO:tasks.ceph.mds.b.smithi070.stderr: 2024-04-22T06:32:55.039 INFO:tasks.ceph.mds.b.smithi070.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T06:32:55.039 INFO:tasks.ceph.mds.b.smithi070.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f8396e3e6f0] 2024-04-22T06:32:55.039 INFO:tasks.ceph.mds.b.smithi070.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f8396e8b94c] 2024-04-22T06:32:55.039 INFO:tasks.ceph.mds.b.smithi070.stderr: 3: raise() 2024-04-22T06:32:55.039 INFO:tasks.ceph.mds.b.smithi070.stderr: 4: abort() 2024-04-22T06:32:55.039 INFO:tasks.ceph.mds.b.smithi070.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f8397781fb6] 2024-04-22T06:32:55.039 INFO:tasks.ceph.mds.b.smithi070.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f839778211a] 2024-04-22T06:32:55.040 INFO:tasks.ceph.mds.b.smithi070.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x55faf4808fd2] 2024-04-22T06:32:55.040 INFO:tasks.ceph.mds.b.smithi070.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x55faf4809091] 2024-04-22T06:32:55.040 INFO:tasks.ceph.mds.b.smithi070.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x55faf47c3a96] 2024-04-22T06:32:55.040 INFO:tasks.ceph.mds.b.smithi070.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x55faf47c39f6] 2024-04-22T06:32:55.040 INFO:tasks.ceph.mds.b.smithi070.stderr: 11: ceph-mds(+0x48e3cf) [0x55faf49613cf] 2024-04-22T06:32:55.040 INFO:tasks.ceph.mds.b.smithi070.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x55faf49b1c1b] 2024-04-22T06:32:55.040 INFO:tasks.ceph.mds.b.smithi070.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7f8396e89c02] 2024-04-22T06:32:55.040 INFO:tasks.ceph.mds.b.smithi070.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7f8396f0ec40] 2024-04-22T06:32:55.040 INFO:tasks.ceph.mds.b.smithi070.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T06:32:55.040 INFO:tasks.ceph.mds.b.smithi070.stderr: 2024-04-22T06:32:55.232 INFO:tasks.ceph.mds.b.smithi070.stderr:daemon-helper: command crashed with signal 6 2024-04-22T06:32:55.822 INFO:tasks.ceph.mds.a.smithi044.stderr:daemon-helper: command crashed with signal 6 2024-04-22T06:32:55.822 INFO:tasks.ceph.mds.c.smithi044.stderr:daemon-helper: command crashed with signal 6 2024-04-22T06:32:55.828 INFO:tasks.ceph.mds.e.smithi044.stderr:daemon-helper: command crashed with signal 6 2024-04-22T06:32:59.948 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-04-22T06:32:59.948 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-04-22T06:32:59.949 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~0s 2024-04-22T06:32:59.949 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-04-22T06:33:06.461 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~7s 2024-04-22T06:33:06.462 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~7s 2024-04-22T06:33:06.462 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~7s 2024-04-22T06:33:06.462 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~7s 2024-04-22T06:33:07.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.141+0000 7fd14cff9640 1 -- 172.21.15.44:0/3200189522 <== mds.0 v2:172.21.15.70:6837/2915150040 1 ==== command_reply(tid 0: -125 ) ==== 8+0+587 (crc 0 0 0) 0x55d9f832ef20 con 0x55d9f840a8f0 2024-04-22T06:33:07.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.141+0000 7fd14cff9640 10 client.5319 handle_command_reply: tid=0 2024-04-22T06:33:07.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.141+0000 7fd1727db740 1 client.5319 shutdown 2024-04-22T06:33:07.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.141+0000 7fd1727db740 1 -- 172.21.15.44:0/3200189522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9f83fb7d0 msgr2=0x55d9f8477460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:07.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.141+0000 7fd1727db740 1 --2- 172.21.15.44:0/3200189522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9f83fb7d0 0x55d9f8477460 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd15c00cfd0 tx=0x7fd15c00a000 comp rx=0 tx=0).stop 2024-04-22T06:33:07.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.141+0000 7fd1727db740 1 -- 172.21.15.44:0/3200189522 shutdown_connections 2024-04-22T06:33:07.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.141+0000 7fd1727db740 1 --2- 172.21.15.44:0/3200189522 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d9f8408b60 0x55d9f84779a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.141+0000 7fd1727db740 1 --2- 172.21.15.44:0/3200189522 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x55d9f840a8f0 0x55d9f8423c30 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.142+0000 7fd1727db740 1 --2- 172.21.15.44:0/3200189522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9f83fb7d0 0x55d9f8477460 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.142+0000 7fd14cff9640 0 client.5319 ms_handle_reset on v2:172.21.15.70:6837/2915150040 2024-04-22T06:33:07.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.142+0000 7fd1727db740 1 --2- 172.21.15.44:0/3200189522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d9f83bcfe0 0x55d9f83d88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.142+0000 7fd1727db740 1 -- 172.21.15.44:0/3200189522 >> 172.21.15.44:0/3200189522 conn(0x55d9f83ba1b0 msgr2=0x55d9f83b5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:33:07.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.143+0000 7fd1727db740 1 -- 172.21.15.44:0/3200189522 shutdown_connections 2024-04-22T06:33:07.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.143+0000 7fd1727db740 1 -- 172.21.15.44:0/3200189522 wait complete. 2024-04-22T06:33:07.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.143+0000 7fd1727db740 20 client.5319 trim_cache size 0 max 16384 2024-04-22T06:33:07.142 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-22T06:33:07.142 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 19, 2024-04-22T06:33:07.143 INFO:teuthology.orchestra.run.smithi044.stdout: "leader": 4308, 2024-04-22T06:33:07.143 INFO:teuthology.orchestra.run.smithi044.stdout: "set_version": 4, 2024-04-22T06:33:07.143 INFO:teuthology.orchestra.run.smithi044.stdout: "sets": { 2024-04-22T06:33:07.143 INFO:teuthology.orchestra.run.smithi044.stdout: "ab138efc": { 2024-04-22T06:33:07.143 INFO:teuthology.orchestra.run.smithi044.stdout: "version": 4, 2024-04-22T06:33:07.143 INFO:teuthology.orchestra.run.smithi044.stdout: "age_ref": 0.0, 2024-04-22T06:33:07.143 INFO:teuthology.orchestra.run.smithi044.stdout: "state": { 2024-04-22T06:33:07.143 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "CANCELED", 2024-04-22T06:33:07.143 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.0 2024-04-22T06:33:07.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-22T06:33:07.143 INFO:teuthology.orchestra.run.smithi044.stdout: "timeout": 60.0, 2024-04-22T06:33:07.143 INFO:teuthology.orchestra.run.smithi044.stdout: "expiration": 86.1, 2024-04-22T06:33:07.143 INFO:teuthology.orchestra.run.smithi044.stdout: "members": { 2024-04-22T06:33:07.143 INFO:teuthology.orchestra.run.smithi044.stdout: "file:/": { 2024-04-22T06:33:07.144 INFO:teuthology.orchestra.run.smithi044.stdout: "excluded": true, 2024-04-22T06:33:07.144 INFO:teuthology.orchestra.run.smithi044.stdout: "state": { 2024-04-22T06:33:07.144 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "QUIESCED", 2024-04-22T06:33:07.144 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 38.8 2024-04-22T06:33:07.144 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-22T06:33:07.144 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-22T06:33:07.144 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-22T06:33:07.144 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-22T06:33:07.144 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-22T06:33:07.144 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-22T06:33:07.144 INFO:teuthology.orchestra.run.smithi044.stderr:Error ECANCELED: 2024-04-22T06:33:07.146 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-04-22T06:33:07.147 WARNING:tasks.quiescer.fs.[cephfs]:Quiesce set got cancelled (cap = 10).Won't raise an error since this could be a failover, will wait for the next quiesce attempt 2024-04-22T06:33:07.147 INFO:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.1 2024-04-22T06:33:07.147 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T06:33:07.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.259+0000 7f8794ce0640 1 Processor -- start 2024-04-22T06:33:07.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.259+0000 7f8794ce0640 1 -- start start 2024-04-22T06:33:07.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.260+0000 7f8794ce0640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f879015f0d0 0x7f87900d0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:07.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.260+0000 7f8794ce0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f87900d1400 0x7f87900cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:07.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.260+0000 7f8794ce0640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f879015b200 con 0x7f8790155240 2024-04-22T06:33:07.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.260+0000 7f8794ce0640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f879015c060 con 0x7f87900d1400 2024-04-22T06:33:07.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.260+0000 7f8794ce0640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f879015b4e0 con 0x7f879015f0d0 2024-04-22T06:33:07.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.260+0000 7f878e59c640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f8790155240 0x7f87900cb320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:53112/0 (socket says 172.21.15.44:53112) 2024-04-22T06:33:07.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.260+0000 7f878e59c640 1 -- 172.21.15.44:0/1504089859 learned_addr learned my addr 172.21.15.44:0/1504089859 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:33:07.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.260+0000 7f878ed9d640 1 --2- 172.21.15.44:0/1504089859 >> v2:172.21.15.70:3300/0 conn(0x7f87900d1400 0x7f87900cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:07.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.260+0000 7f878dd9b640 1 --2- 172.21.15.44:0/1504089859 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f879015f0d0 0x7f87900d0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:07.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.260+0000 7f878d59a640 1 -- 172.21.15.44:0/1504089859 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3944601515 0 0) 0x7f879015b200 con 0x7f8790155240 2024-04-22T06:33:07.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.260+0000 7f878d59a640 1 -- 172.21.15.44:0/1504089859 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8778003520 con 0x7f8790155240 2024-04-22T06:33:07.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.260+0000 7f878d59a640 1 -- 172.21.15.44:0/1504089859 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3620736851 0 0) 0x7f8778003520 con 0x7f8790155240 2024-04-22T06:33:07.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.260+0000 7f878ed9d640 1 -- 172.21.15.44:0/1504089859 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f879015f0d0 msgr2=0x7f87900d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:33:07.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.260+0000 7f878ed9d640 1 --2- 172.21.15.44:0/1504089859 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f879015f0d0 0x7f87900d0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.260+0000 7f878ed9d640 1 -- 172.21.15.44:0/1504089859 >> v1:172.21.15.44:6789/0 conn(0x7f8790155240 legacy=0x7f87900cb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:07.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.260+0000 7f878ed9d640 1 -- 172.21.15.44:0/1504089859 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f879015b7c0 con 0x7f87900d1400 2024-04-22T06:33:07.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.260+0000 7f878dd9b640 1 --2- 172.21.15.44:0/1504089859 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f879015f0d0 0x7f87900d0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:33:07.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.261+0000 7f878ed9d640 1 --2- 172.21.15.44:0/1504089859 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87900d1400 0x7f87900cc480 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8788009d90 tx=0x7f878802bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fed621df3d58223 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:07.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.261+0000 7f878d59a640 1 -- 172.21.15.44:0/1504089859 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8788039070 con 0x7f87900d1400 2024-04-22T06:33:07.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.261+0000 7f878d59a640 1 -- 172.21.15.44:0/1504089859 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f878802c8a0 con 0x7f87900d1400 2024-04-22T06:33:07.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.262+0000 7f878d59a640 1 -- 172.21.15.44:0/1504089859 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8780001670 con 0x7f87900d1400 2024-04-22T06:33:07.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.262+0000 7f8794ce0640 1 -- 172.21.15.44:0/1504089859 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87900d1400 msgr2=0x7f87900cc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:07.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.262+0000 7f8794ce0640 1 --2- 172.21.15.44:0/1504089859 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87900d1400 0x7f87900cc480 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8788009d90 tx=0x7f878802bd70 comp rx=0 tx=0).stop 2024-04-22T06:33:07.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.262+0000 7f8794ce0640 1 -- 172.21.15.44:0/1504089859 shutdown_connections 2024-04-22T06:33:07.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.262+0000 7f8794ce0640 1 --2- 172.21.15.44:0/1504089859 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87900d1400 0x7f87900cc480 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.262+0000 7f8794ce0640 1 --2- 172.21.15.44:0/1504089859 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f879015f0d0 0x7f87900d0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.262+0000 7f8794ce0640 1 -- 172.21.15.44:0/1504089859 >> 172.21.15.44:0/1504089859 conn(0x7f87900750d0 msgr2=0x7f87900754e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:33:07.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.262+0000 7f8794ce0640 1 -- 172.21.15.44:0/1504089859 shutdown_connections 2024-04-22T06:33:07.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.262+0000 7f8794ce0640 1 -- 172.21.15.44:0/1504089859 wait complete. 2024-04-22T06:33:07.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.263+0000 7f8794ce0640 1 Processor -- start 2024-04-22T06:33:07.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.263+0000 7f8794ce0640 1 -- start start 2024-04-22T06:33:07.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.264+0000 7f8794ce0640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87900d1400 0x7f87900cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:07.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.264+0000 7f8794ce0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8790155240 0x7f8790158d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:07.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.264+0000 7f8794ce0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f879015f0d0 0x7f879015c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:07.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.264+0000 7f8794ce0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f879015f4f0 con 0x7f879015f0d0 2024-04-22T06:33:07.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.264+0000 7f8794ce0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f879015bd80 con 0x7f8790155240 2024-04-22T06:33:07.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.264+0000 7f8794ce0640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8790155620 con 0x7f87900d1400 2024-04-22T06:33:07.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.264+0000 7f878ed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f879015f0d0 0x7f879015c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:07.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.264+0000 7f878ed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f879015f0d0 0x7f879015c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60310/0 (socket says 172.21.15.44:60310) 2024-04-22T06:33:07.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.264+0000 7f878ed9d640 1 -- 172.21.15.44:0/4217494410 learned_addr learned my addr 172.21.15.44:0/4217494410 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:33:07.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.264+0000 7f878e59c640 1 --2- 172.21.15.44:0/4217494410 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87900d1400 0x7f87900cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:07.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.264+0000 7f878dd9b640 1 --2- 172.21.15.44:0/4217494410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8790155240 0x7f8790158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:07.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.265+0000 7f878ed9d640 1 -- 172.21.15.44:0/4217494410 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87900d1400 msgr2=0x7f87900cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:07.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.265+0000 7f878ed9d640 1 --2- 172.21.15.44:0/4217494410 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87900d1400 0x7f87900cfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.265+0000 7f878ed9d640 1 -- 172.21.15.44:0/4217494410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8790155240 msgr2=0x7f8790158d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:07.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.265+0000 7f878ed9d640 1 --2- 172.21.15.44:0/4217494410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8790155240 0x7f8790158d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.265+0000 7f878ed9d640 1 -- 172.21.15.44:0/4217494410 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8790159600 con 0x7f879015f0d0 2024-04-22T06:33:07.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.265+0000 7f878dd9b640 1 --2- 172.21.15.44:0/4217494410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8790155240 0x7f8790158d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:33:07.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.265+0000 7f878ed9d640 1 --2- 172.21.15.44:0/4217494410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f879015f0d0 0x7f879015c2b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8788037040 tx=0x7f8788004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:07.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.265+0000 7f877effd640 1 -- 172.21.15.44:0/4217494410 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8788039050 con 0x7f879015f0d0 2024-04-22T06:33:07.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.265+0000 7f877effd640 1 -- 172.21.15.44:0/4217494410 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8788045070 con 0x7f879015f0d0 2024-04-22T06:33:07.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.265+0000 7f8794ce0640 1 -- 172.21.15.44:0/4217494410 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87901f4b80 con 0x7f879015f0d0 2024-04-22T06:33:07.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.265+0000 7f877effd640 1 -- 172.21.15.44:0/4217494410 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8788009450 con 0x7f879015f0d0 2024-04-22T06:33:07.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.265+0000 7f8794ce0640 1 -- 172.21.15.44:0/4217494410 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f879015aca0 con 0x7f879015f0d0 2024-04-22T06:33:07.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.266+0000 7f8794ce0640 1 -- 172.21.15.44:0/4217494410 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8790068cc0 con 0x7f879015f0d0 2024-04-22T06:33:07.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.267+0000 7f877effd640 1 -- 172.21.15.44:0/4217494410 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8788040690 con 0x7f879015f0d0 2024-04-22T06:33:07.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.267+0000 7f877effd640 1 --2- 172.21.15.44:0/4217494410 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f877006c8f0 0x7f877006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:07.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.267+0000 7f877effd640 1 -- 172.21.15.44:0/4217494410 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6821+0+0 (secure 0 0 0) 0x7f87880ae880 con 0x7f879015f0d0 2024-04-22T06:33:07.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.267+0000 7f878e59c640 1 --2- 172.21.15.44:0/4217494410 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f877006c8f0 0x7f877006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:07.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.268+0000 7f878e59c640 1 --2- 172.21.15.44:0/4217494410 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f877006c8f0 0x7f877006edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f87800013c0 tx=0x7f8780001350 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:07.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.272+0000 7f877effd640 1 -- 172.21.15.44:0/4217494410 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f878807c7c0 con 0x7f879015f0d0 2024-04-22T06:33:07.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.499+0000 7f8794ce0640 1 -- 172.21.15.44:0/4217494410 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87900cadd0 con 0x7f879015f0d0 2024-04-22T06:33:07.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.499+0000 7f877effd640 1 -- 172.21.15.44:0/4217494410 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+6125 (secure 0 0 0) 0x7f878807c0c0 con 0x7f879015f0d0 2024-04-22T06:33:07.498 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:33:07.498 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T06:31:06.356312+0000","modified":"2024-04-22T06:33:07.139930+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4308,"mds_1":4305,"mds_2":4293,"mds_3":4290,"mds_4":4253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4253":{"gid":4253,"name":"f","rank":4,"incarnation":19,"state":"up:replay","state_seq":1,"addr":"172.21.15.70:6836/1779271860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":1779271860},{"type":"v1","addr":"172.21.15.70:6836","nonce":1779271860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4290":{"gid":4290,"name":"a","rank":3,"incarnation":15,"state":"up:active","state_seq":14,"addr":"172.21.15.44:6835/2383621014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2383621014},{"type":"v1","addr":"172.21.15.44:6835","nonce":2383621014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"e","rank":2,"incarnation":13,"state":"up:active","state_seq":12,"addr":"172.21.15.44:6837/915960587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":915960587},{"type":"v1","addr":"172.21.15.44:6837","nonce":915960587}]},"join_fscid":-1,"laggy_since":"2024-04-22T06:33:07.139644+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4305":{"gid":4305,"name":"b","rank":1,"incarnation":9,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6838/924955785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":924955785},{"type":"v1","addr":"172.21.15.70:6838","nonce":924955785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4308":{"gid":4308,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.70:6839/2915150040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6837","nonce":2915150040},{"type":"v1","addr":"172.21.15.70:6839","nonce":2915150040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4308,"qdb_cluster":[4290,4293,4305,4308]},"id":1}]} 2024-04-22T06:33:07.499 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 19 2024-04-22T06:33:07.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.502+0000 7f8794ce0640 1 -- 172.21.15.44:0/4217494410 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f877006c8f0 msgr2=0x7f877006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:07.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.502+0000 7f8794ce0640 1 --2- 172.21.15.44:0/4217494410 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f877006c8f0 0x7f877006edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f87800013c0 tx=0x7f8780001350 comp rx=0 tx=0).stop 2024-04-22T06:33:07.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.502+0000 7f8794ce0640 1 -- 172.21.15.44:0/4217494410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f879015f0d0 msgr2=0x7f879015c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:07.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.502+0000 7f8794ce0640 1 --2- 172.21.15.44:0/4217494410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f879015f0d0 0x7f879015c2b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8788037040 tx=0x7f8788004a40 comp rx=0 tx=0).stop 2024-04-22T06:33:07.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.502+0000 7f8794ce0640 1 -- 172.21.15.44:0/4217494410 shutdown_connections 2024-04-22T06:33:07.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.502+0000 7f8794ce0640 1 --2- 172.21.15.44:0/4217494410 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f877006c8f0 0x7f877006edb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.502+0000 7f8794ce0640 1 --2- 172.21.15.44:0/4217494410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f879015f0d0 0x7f879015c2b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.502+0000 7f8794ce0640 1 --2- 172.21.15.44:0/4217494410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8790155240 0x7f8790158d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.502+0000 7f8794ce0640 1 --2- 172.21.15.44:0/4217494410 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87900d1400 0x7f87900cfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.502+0000 7f8794ce0640 1 -- 172.21.15.44:0/4217494410 >> 172.21.15.44:0/4217494410 conn(0x7f87900750d0 msgr2=0x7f87900c6480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:33:07.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.502+0000 7f8794ce0640 1 -- 172.21.15.44:0/4217494410 shutdown_connections 2024-04-22T06:33:07.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.502+0000 7f8794ce0640 1 -- 172.21.15.44:0/4217494410 wait complete. 2024-04-22T06:33:07.512 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.4308 quiesce db / --timeout 60 --expiration 86.1 --await' 2024-04-22T06:33:07.512 DEBUG:teuthology.orchestra.run.smithi044:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4308 quiesce db / --timeout 60 --expiration 86.1 --await 2024-04-22T06:33:07.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.631+0000 7fdaf15fd640 1 Processor -- start 2024-04-22T06:33:07.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.631+0000 7fdaf15fd640 1 -- start start 2024-04-22T06:33:07.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.631+0000 7fdaf15fd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdaec15ec20 0x7fdaec0ce5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:07.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.631+0000 7fdaf15fd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdaec15f0b0 0x7fdaec0ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:07.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.631+0000 7fdaf15fd640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaec1556f0 con 0x7fdaec155310 2024-04-22T06:33:07.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.631+0000 7fdaf15fd640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fdaec0c9ab0 con 0x7fdaec15ec20 2024-04-22T06:33:07.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.631+0000 7fdaf15fd640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdaec15b3b0 con 0x7fdaec15f0b0 2024-04-22T06:33:07.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.632+0000 7fdaeaffd640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fdaec155310 0x7fdaec073930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:53118/0 (socket says 172.21.15.44:53118) 2024-04-22T06:33:07.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.632+0000 7fdaeaffd640 1 -- 172.21.15.44:0/3620075259 learned_addr learned my addr 172.21.15.44:0/3620075259 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:33:07.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.632+0000 7fdaeb7fe640 1 --2- 172.21.15.44:0/3620075259 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdaec15f0b0 0x7fdaec0ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:07.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.632+0000 7fdaea7fc640 1 --2- 172.21.15.44:0/3620075259 >> v2:172.21.15.70:3300/0 conn(0x7fdaec15ec20 0x7fdaec0ce5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:07.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.632+0000 7fdae9ffb640 1 -- 172.21.15.44:0/3620075259 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3845292337 0 0) 0x7fdaec1556f0 con 0x7fdaec155310 2024-04-22T06:33:07.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.632+0000 7fdae9ffb640 1 -- 172.21.15.44:0/3620075259 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdad4003520 con 0x7fdaec155310 2024-04-22T06:33:07.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.632+0000 7fdaeb7fe640 1 -- 172.21.15.44:0/3620075259 >> v2:172.21.15.70:3300/0 conn(0x7fdaec15ec20 msgr2=0x7fdaec0ce5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:33:07.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.632+0000 7fdae9ffb640 1 -- 172.21.15.44:0/3620075259 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 806718679 0 0) 0x7fdad4003520 con 0x7fdaec155310 2024-04-22T06:33:07.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.632+0000 7fdaeb7fe640 1 --2- 172.21.15.44:0/3620075259 >> v2:172.21.15.70:3300/0 conn(0x7fdaec15ec20 0x7fdaec0ce5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.632+0000 7fdaeb7fe640 1 -- 172.21.15.44:0/3620075259 >> v1:172.21.15.44:6789/0 conn(0x7fdaec155310 legacy=0x7fdaec073930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:07.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.632+0000 7fdaeb7fe640 1 -- 172.21.15.44:0/3620075259 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaec15b690 con 0x7fdaec15f0b0 2024-04-22T06:33:07.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.633+0000 7fdaeb7fe640 1 --2- 172.21.15.44:0/3620075259 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdaec15f0b0 0x7fdaec0ceb10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdae4009d90 tx=0x7fdae402be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71b87c85c70e4da1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:07.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.633+0000 7fdae9ffb640 1 -- 172.21.15.44:0/3620075259 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae4039070 con 0x7fdaec15f0b0 2024-04-22T06:33:07.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.633+0000 7fdae9ffb640 1 -- 172.21.15.44:0/3620075259 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdae402c930 con 0x7fdaec15f0b0 2024-04-22T06:33:07.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.633+0000 7fdae9ffb640 1 -- 172.21.15.44:0/3620075259 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc001670 con 0x7fdaec15f0b0 2024-04-22T06:33:07.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.634+0000 7fdaf15fd640 1 -- 172.21.15.44:0/3620075259 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdaec15f0b0 msgr2=0x7fdaec0ceb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:07.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.634+0000 7fdaf15fd640 1 --2- 172.21.15.44:0/3620075259 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdaec15f0b0 0x7fdaec0ceb10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdae4009d90 tx=0x7fdae402be00 comp rx=0 tx=0).stop 2024-04-22T06:33:07.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.634+0000 7fdaf15fd640 1 -- 172.21.15.44:0/3620075259 shutdown_connections 2024-04-22T06:33:07.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.634+0000 7fdaf15fd640 1 --2- 172.21.15.44:0/3620075259 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdaec15f0b0 0x7fdaec0ceb10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.634+0000 7fdaf15fd640 1 --2- 172.21.15.44:0/3620075259 >> v2:172.21.15.70:3300/0 conn(0x7fdaec15ec20 0x7fdaec0ce5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.634+0000 7fdaf15fd640 1 -- 172.21.15.44:0/3620075259 >> 172.21.15.44:0/3620075259 conn(0x7fdaec074cb0 msgr2=0x7fdaec0750c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:33:07.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.634+0000 7fdaf15fd640 1 -- 172.21.15.44:0/3620075259 shutdown_connections 2024-04-22T06:33:07.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.634+0000 7fdaf15fd640 1 -- 172.21.15.44:0/3620075259 wait complete. 2024-04-22T06:33:07.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.635+0000 7fdaf15fd640 1 Processor -- start 2024-04-22T06:33:07.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.635+0000 7fdaf15fd640 1 -- start start 2024-04-22T06:33:07.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.636+0000 7fdaf15fd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdaec155310 0x7fdaec1e4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:07.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.636+0000 7fdaf15fd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdaec15ec20 0x7fdaec1e60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:07.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.636+0000 7fdaf15fd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdaec15f0b0 0x7fdaec1e65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:07.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.636+0000 7fdaf15fd640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdaec15e700 con 0x7fdaec15ec20 2024-04-22T06:33:07.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.636+0000 7fdaf15fd640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fdaec15bc50 con 0x7fdaec15f0b0 2024-04-22T06:33:07.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.636+0000 7fdaf15fd640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdaec06d990 con 0x7fdaec155310 2024-04-22T06:33:07.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.636+0000 7fdaea7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdaec15ec20 0x7fdaec1e60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:07.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.636+0000 7fdaea7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdaec15ec20 0x7fdaec1e60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60316/0 (socket says 172.21.15.44:60316) 2024-04-22T06:33:07.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.636+0000 7fdaea7fc640 1 -- 172.21.15.44:0/498650696 learned_addr learned my addr 172.21.15.44:0/498650696 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:33:07.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.636+0000 7fdaeaffd640 1 --2- 172.21.15.44:0/498650696 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdaec155310 0x7fdaec1e4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:07.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.636+0000 7fdaeb7fe640 1 --2- 172.21.15.44:0/498650696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdaec15f0b0 0x7fdaec1e65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:07.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.637+0000 7fdaea7fc640 1 -- 172.21.15.44:0/498650696 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdaec155310 msgr2=0x7fdaec1e4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:07.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.637+0000 7fdaea7fc640 1 --2- 172.21.15.44:0/498650696 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdaec155310 0x7fdaec1e4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.637+0000 7fdaea7fc640 1 -- 172.21.15.44:0/498650696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdaec15f0b0 msgr2=0x7fdaec1e65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:07.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.637+0000 7fdaea7fc640 1 --2- 172.21.15.44:0/498650696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdaec15f0b0 0x7fdaec1e65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.637+0000 7fdaea7fc640 1 -- 172.21.15.44:0/498650696 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaec1f0ab0 con 0x7fdaec15ec20 2024-04-22T06:33:07.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.637+0000 7fdaea7fc640 1 --2- 172.21.15.44:0/498650696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdaec15ec20 0x7fdaec1e60a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdae000e870 tx=0x7fdae000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:07.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.637+0000 7fdad37fe640 1 -- 172.21.15.44:0/498650696 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae0017070 con 0x7fdaec15ec20 2024-04-22T06:33:07.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.637+0000 7fdad37fe640 1 -- 172.21.15.44:0/498650696 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdae000c940 con 0x7fdaec15ec20 2024-04-22T06:33:07.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.637+0000 7fdaf15fd640 1 -- 172.21.15.44:0/498650696 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaec1e5a40 con 0x7fdaec15ec20 2024-04-22T06:33:07.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.637+0000 7fdad37fe640 1 -- 172.21.15.44:0/498650696 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae000cd30 con 0x7fdaec15ec20 2024-04-22T06:33:07.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.637+0000 7fdaf15fd640 1 -- 172.21.15.44:0/498650696 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaec1f1050 con 0x7fdaec15ec20 2024-04-22T06:33:07.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.639+0000 7fdad37fe640 1 -- 172.21.15.44:0/498650696 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdae001a070 con 0x7fdaec15ec20 2024-04-22T06:33:07.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.640+0000 7fdad37fe640 1 --2- 172.21.15.44:0/498650696 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fdad806c8f0 0x7fdad806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:07.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.640+0000 7fdad37fe640 1 -- 172.21.15.44:0/498650696 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6821+0+0 (secure 0 0 0) 0x7fdae008be70 con 0x7fdaec15ec20 2024-04-22T06:33:07.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.640+0000 7fdaeaffd640 1 --2- 172.21.15.44:0/498650696 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fdad806c8f0 0x7fdad806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:07.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.641+0000 7fdaeaffd640 1 --2- 172.21.15.44:0/498650696 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fdad806c8f0 0x7fdad806edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdadc001320 tx=0x7fdadc0012b0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:07.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.641+0000 7fdaf41e0740 1 Processor -- start 2024-04-22T06:33:07.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.642+0000 7fdaf41e0740 1 -- start start 2024-04-22T06:33:07.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.643+0000 7fdaf41e0740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5559639a1200 0x5559639a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:07.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.643+0000 7fdaf41e0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559639eb080 0x5559639a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:07.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.643+0000 7fdaf41e0740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55596399fc60 0x5559639e0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:07.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.643+0000 7fdaf41e0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5559639020b0 con 0x5559639eb080 2024-04-22T06:33:07.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.643+0000 7fdaf41e0740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5559638b3680 con 0x55596399fc60 2024-04-22T06:33:07.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.643+0000 7fdaf41e0740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5559638bfc60 con 0x5559639a1200 2024-04-22T06:33:07.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.643+0000 7fdaea7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559639eb080 0x5559639a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:07.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.643+0000 7fdaea7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559639eb080 0x5559639a1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60326/0 (socket says 172.21.15.44:60326) 2024-04-22T06:33:07.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.643+0000 7fdaea7fc640 1 -- 172.21.15.44:0/2070071225 learned_addr learned my addr 172.21.15.44:0/2070071225 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:33:07.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.643+0000 7fdaeaffd640 1 --2- 172.21.15.44:0/2070071225 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5559639a1200 0x5559639a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:07.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.643+0000 7fdaeb7fe640 1 --2- 172.21.15.44:0/2070071225 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55596399fc60 0x5559639e0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:07.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.643+0000 7fdaea7fc640 1 -- 172.21.15.44:0/2070071225 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5559639a1200 msgr2=0x5559639a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:33:07.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.643+0000 7fdaea7fc640 1 --2- 172.21.15.44:0/2070071225 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5559639a1200 0x5559639a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.643+0000 7fdaea7fc640 1 -- 172.21.15.44:0/2070071225 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55596399fc60 msgr2=0x5559639e0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:33:07.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.643+0000 7fdaea7fc640 1 --2- 172.21.15.44:0/2070071225 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55596399fc60 0x5559639e0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.643+0000 7fdaea7fc640 1 -- 172.21.15.44:0/2070071225 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559636e4c90 con 0x5559639eb080 2024-04-22T06:33:07.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.643+0000 7fdaea7fc640 1 --2- 172.21.15.44:0/2070071225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559639eb080 0x5559639a1b20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fdae0061d50 tx=0x7fdae00590b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb46453e57f702f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:07.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.643+0000 7fdad0ff9640 1 -- 172.21.15.44:0/2070071225 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae0021070 con 0x5559639eb080 2024-04-22T06:33:07.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.643+0000 7fdad0ff9640 1 -- 172.21.15.44:0/2070071225 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdae0062280 con 0x5559639eb080 2024-04-22T06:33:07.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.643+0000 7fdad0ff9640 1 -- 172.21.15.44:0/2070071225 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae0055dc0 con 0x5559639eb080 2024-04-22T06:33:07.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.644+0000 7fdaf41e0740 1 -- 172.21.15.44:0/2070071225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559639eb080 msgr2=0x5559639a1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:07.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.644+0000 7fdaf41e0740 1 --2- 172.21.15.44:0/2070071225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559639eb080 0x5559639a1b20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fdae0061d50 tx=0x7fdae00590b0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.644+0000 7fdaf41e0740 1 -- 172.21.15.44:0/2070071225 shutdown_connections 2024-04-22T06:33:07.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.644+0000 7fdaf41e0740 1 --2- 172.21.15.44:0/2070071225 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55596399fc60 0x5559639e0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.644+0000 7fdaf41e0740 1 --2- 172.21.15.44:0/2070071225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559639eb080 0x5559639a1b20 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.644+0000 7fdaf41e0740 1 --2- 172.21.15.44:0/2070071225 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5559639a1200 0x5559639a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.644+0000 7fdaf41e0740 1 -- 172.21.15.44:0/2070071225 >> 172.21.15.44:0/2070071225 conn(0x55596399dc00 msgr2=0x5559639a02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:33:07.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.645+0000 7fdaf41e0740 1 -- 172.21.15.44:0/2070071225 shutdown_connections 2024-04-22T06:33:07.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.645+0000 7fdaf41e0740 1 -- 172.21.15.44:0/2070071225 wait complete. 2024-04-22T06:33:07.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.645+0000 7fdaf41e0740 1 -- start start 2024-04-22T06:33:07.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.645+0000 7fdaf41e0740 1 Processor -- start 2024-04-22T06:33:07.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.646+0000 7fdaf41e0740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5559639a1200 0x5559639e6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:07.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.646+0000 7fdaf41e0740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559639eb080 0x5559639c7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:07.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.646+0000 7fdaf41e0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559639dbf80 0x5559639f5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:07.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.646+0000 7fdaf41e0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x555963905400 con 0x5559639dbf80 2024-04-22T06:33:07.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.646+0000 7fdaf41e0740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5559638c1840 con 0x5559639eb080 2024-04-22T06:33:07.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.646+0000 7fdaf41e0740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55596390a720 con 0x5559639a1200 2024-04-22T06:33:07.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.646+0000 7fdaf41e0740 10 client.0 _ll_register_callbacks cb 0x555963912ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T06:33:07.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.646+0000 7fdaeb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559639dbf80 0x5559639f5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:07.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.646+0000 7fdaeaffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5559639a1200 0x5559639e6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:07.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.646+0000 7fdaea7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559639eb080 0x5559639c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:07.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.646+0000 7fdaeb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559639dbf80 0x5559639f5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60332/0 (socket says 172.21.15.44:60332) 2024-04-22T06:33:07.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.646+0000 7fdaeb7fe640 1 -- 172.21.15.44:0/1285724855 learned_addr learned my addr 172.21.15.44:0/1285724855 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:33:07.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.647+0000 7fdaeb7fe640 1 -- 172.21.15.44:0/1285724855 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5559639a1200 msgr2=0x5559639e6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:33:07.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.647+0000 7fdaeb7fe640 1 --2- 172.21.15.44:0/1285724855 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5559639a1200 0x5559639e6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.647+0000 7fdaeb7fe640 1 -- 172.21.15.44:0/1285724855 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559639eb080 msgr2=0x5559639c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:33:07.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.647+0000 7fdaeb7fe640 1 --2- 172.21.15.44:0/1285724855 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559639eb080 0x5559639c7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.647+0000 7fdaeb7fe640 1 -- 172.21.15.44:0/1285724855 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555963745420 con 0x5559639dbf80 2024-04-22T06:33:07.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.647+0000 7fdaeaffd640 1 --2- 172.21.15.44:0/1285724855 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5559639a1200 0x5559639e6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:33:07.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.647+0000 7fdaeb7fe640 1 --2- 172.21.15.44:0/1285724855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559639dbf80 0x5559639f5bd0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fdae4009eb0 tx=0x7fdae4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad8041acc8e71c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:07.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.647+0000 7fdac9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T06:33:07.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.647+0000 7fdac9ffb640 1 -- 172.21.15.44:0/1285724855 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae4039040 con 0x5559639dbf80 2024-04-22T06:33:07.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.647+0000 7fdac9ffb640 1 -- 172.21.15.44:0/1285724855 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(28..28 src has 1..28) ==== 6821+0+0 (secure 0 0 0) 0x7fdae4034850 con 0x5559639dbf80 2024-04-22T06:33:07.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.647+0000 7fdaf41e0740 1 -- 172.21.15.44:0/1285724855 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdae0062280 con 0x5559639dbf80 2024-04-22T06:33:07.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.647+0000 7fdac9ffb640 1 -- 172.21.15.44:0/1285724855 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fdae4052050 con 0x5559639dbf80 2024-04-22T06:33:07.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.647+0000 7fdaf41e0740 10 client.5337 fetch_fsmap learned FSMap version 19 2024-04-22T06:33:07.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.647+0000 7fdaf41e0740 1 -- 172.21.15.44:0/1285724855 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x5559635d8580 con 0x5559639dbf80 2024-04-22T06:33:07.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.647+0000 7fdac9ffb640 1 -- 172.21.15.44:0/1285724855 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdae0062280 con 0x5559639dbf80 2024-04-22T06:33:07.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.647+0000 7fdac9ffb640 1 -- 172.21.15.44:0/1285724855 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 19) ==== 4179+0+0 (secure 0 0 0) 0x7fdae4055020 con 0x5559639dbf80 2024-04-22T06:33:07.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.648+0000 7fdaf41e0740 10 client.5337 fetch_fsmap finished waiting for FSMap version 19 2024-04-22T06:33:07.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.648+0000 7fdaf41e0740 10 client.5337 resolve_mds: validated gid 4308 aka daemon mds.d 2024-04-22T06:33:07.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.648+0000 7fdaf41e0740 20 client.5337 populate_metadata read hostname 'smithi044' 2024-04-22T06:33:07.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.648+0000 7fdaf41e0740 1 --2- 172.21.15.44:0/1285724855 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x5559639c3ac0 0x5559639c3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:07.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.648+0000 7fdaf41e0740 4 client.5337 mds_command: new command op to 4308 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T06:33:07.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.648+0000 7fdaf41e0740 1 -- 172.21.15.44:0/1285724855 --> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555963913b10 con 0x5559639c3ac0 2024-04-22T06:33:07.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.648+0000 7fdaeaffd640 1 --2- 172.21.15.44:0/1285724855 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x5559639c3ac0 0x5559639c3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:07.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.648+0000 7fdaeaffd640 1 --2- 172.21.15.44:0/1285724855 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x5559639c3ac0 0x5559639c3ea0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7df4a9509b92b94 server_cookie=73300824600ec673 in_seq=0 out_seq=0 2024-04-22T06:33:07.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.648+0000 7fdac9ffb640 10 client.5337 ms_handle_connect on v2:172.21.15.70:6837/2915150040 2024-04-22T06:33:07.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.649+0000 7fdac9ffb640 1 -- 172.21.15.44:0/1285724855 <== mds.0 v2:172.21.15.70:6837/2915150040 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555963913b10 con 0x5559639c3ac0 2024-04-22T06:33:07.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.649+0000 7fdac9ffb640 10 client.5337 handle_command_reply: tid=0 2024-04-22T06:33:07.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.649+0000 7fdaf41e0740 1 client.5337 shutdown 2024-04-22T06:33:07.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.649+0000 7fdaf41e0740 1 -- 172.21.15.44:0/1285724855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559639dbf80 msgr2=0x5559639f5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:07.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.649+0000 7fdaf41e0740 1 --2- 172.21.15.44:0/1285724855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559639dbf80 0x5559639f5bd0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fdae4009eb0 tx=0x7fdae4004ab0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.649+0000 7fdaf41e0740 1 -- 172.21.15.44:0/1285724855 shutdown_connections 2024-04-22T06:33:07.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.649+0000 7fdaf41e0740 1 --2- 172.21.15.44:0/1285724855 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x5559639c3ac0 0x5559639c3ea0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.649+0000 7fdaf41e0740 1 --2- 172.21.15.44:0/1285724855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559639dbf80 0x5559639f5bd0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.649+0000 7fdaf41e0740 1 --2- 172.21.15.44:0/1285724855 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559639eb080 0x5559639c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.649+0000 7fdac9ffb640 0 client.5337 ms_handle_reset on v2:172.21.15.70:6837/2915150040 2024-04-22T06:33:07.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.649+0000 7fdaf41e0740 1 --2- 172.21.15.44:0/1285724855 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5559639a1200 0x5559639e6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.649+0000 7fdaf41e0740 1 -- 172.21.15.44:0/1285724855 >> 172.21.15.44:0/1285724855 conn(0x55596399fc60 msgr2=0x5559639e5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:33:07.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.649+0000 7fdaf41e0740 1 -- 172.21.15.44:0/1285724855 shutdown_connections 2024-04-22T06:33:07.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.649+0000 7fdaf41e0740 1 -- 172.21.15.44:0/1285724855 wait complete. 2024-04-22T06:33:07.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.649+0000 7fdaf41e0740 20 client.5337 trim_cache size 0 max 16384 2024-04-22T06:33:07.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.681+0000 7fdaf41e0740 1 Processor -- start 2024-04-22T06:33:07.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.681+0000 7fdaf41e0740 1 -- start start 2024-04-22T06:33:07.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.682+0000 7fdaf41e0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559639a1200 0x5559639c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:07.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.682+0000 7fdaf41e0740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559639dbf80 0x5559639f5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:07.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.682+0000 7fdaf41e0740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5559639eb080 0x555963a04f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:07.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.682+0000 7fdaf41e0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55596399d9a0 con 0x5559639a1200 2024-04-22T06:33:07.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.682+0000 7fdaf41e0740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fdaec1ff470 con 0x5559639dbf80 2024-04-22T06:33:07.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.682+0000 7fdaf41e0740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdaec1fd930 con 0x5559639eb080 2024-04-22T06:33:07.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.682+0000 7fdaeaffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559639a1200 0x5559639c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:07.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.682+0000 7fdaeaffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559639a1200 0x5559639c5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60334/0 (socket says 172.21.15.44:60334) 2024-04-22T06:33:07.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.682+0000 7fdaeaffd640 1 -- 172.21.15.44:0/3828545550 learned_addr learned my addr 172.21.15.44:0/3828545550 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:33:07.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.682+0000 7fdaeb7fe640 1 --2- 172.21.15.44:0/3828545550 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5559639eb080 0x555963a04f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:07.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.682+0000 7fdaea7fc640 1 --2- 172.21.15.44:0/3828545550 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559639dbf80 0x5559639f5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:07.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.682+0000 7fdaeaffd640 1 -- 172.21.15.44:0/3828545550 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5559639eb080 msgr2=0x555963a04f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:33:07.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.682+0000 7fdaeaffd640 1 --2- 172.21.15.44:0/3828545550 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5559639eb080 0x555963a04f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.682+0000 7fdaeaffd640 1 -- 172.21.15.44:0/3828545550 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559639dbf80 msgr2=0x5559639f5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:33:07.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.683+0000 7fdaeaffd640 1 --2- 172.21.15.44:0/3828545550 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559639dbf80 0x5559639f5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.683+0000 7fdaeaffd640 1 -- 172.21.15.44:0/3828545550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559639ea130 con 0x5559639a1200 2024-04-22T06:33:07.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.683+0000 7fdaeaffd640 1 --2- 172.21.15.44:0/3828545550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559639a1200 0x5559639c5db0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fdadc0155c0 tx=0x7fdadc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a89d19fd21b8b447 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:07.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.683+0000 7fdacbfff640 1 -- 172.21.15.44:0/3828545550 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc0058d0 con 0x5559639a1200 2024-04-22T06:33:07.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.683+0000 7fdacbfff640 1 -- 172.21.15.44:0/3828545550 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdadc01b040 con 0x5559639a1200 2024-04-22T06:33:07.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.683+0000 7fdacbfff640 1 -- 172.21.15.44:0/3828545550 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc005c90 con 0x5559639a1200 2024-04-22T06:33:07.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.684+0000 7fdaf41e0740 1 -- 172.21.15.44:0/3828545550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559639a1200 msgr2=0x5559639c5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:07.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.684+0000 7fdaf41e0740 1 --2- 172.21.15.44:0/3828545550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559639a1200 0x5559639c5db0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fdadc0155c0 tx=0x7fdadc012000 comp rx=0 tx=0).stop 2024-04-22T06:33:07.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.684+0000 7fdaf41e0740 1 -- 172.21.15.44:0/3828545550 shutdown_connections 2024-04-22T06:33:07.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.684+0000 7fdaf41e0740 1 --2- 172.21.15.44:0/3828545550 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5559639eb080 0x555963a04f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.684+0000 7fdaf41e0740 1 --2- 172.21.15.44:0/3828545550 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559639dbf80 0x5559639f5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.684+0000 7fdaf41e0740 1 --2- 172.21.15.44:0/3828545550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559639a1200 0x5559639c5db0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.684+0000 7fdaf41e0740 1 -- 172.21.15.44:0/3828545550 >> 172.21.15.44:0/3828545550 conn(0x55596399fc60 msgr2=0x55596399c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:33:07.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.684+0000 7fdaf41e0740 1 -- 172.21.15.44:0/3828545550 shutdown_connections 2024-04-22T06:33:07.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.684+0000 7fdaf41e0740 1 -- 172.21.15.44:0/3828545550 wait complete. 2024-04-22T06:33:07.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.685+0000 7fdaf41e0740 1 -- start start 2024-04-22T06:33:07.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.685+0000 7fdaf41e0740 1 Processor -- start 2024-04-22T06:33:07.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.686+0000 7fdaf41e0740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559639a1200 0x5559639db7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:07.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.686+0000 7fdaf41e0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559639dbf80 0x555963a55b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:07.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.686+0000 7fdaf41e0740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5559639eb080 0x555963a560a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:07.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.686+0000 7fdaf41e0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdaec1fe000 con 0x5559639dbf80 2024-04-22T06:33:07.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.686+0000 7fdaf41e0740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fdaec1feda0 con 0x5559639a1200 2024-04-22T06:33:07.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.686+0000 7fdaf41e0740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdaec1fe6d0 con 0x5559639eb080 2024-04-22T06:33:07.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.686+0000 7fdaf41e0740 10 client.0 _ll_register_callbacks cb 0x7fdad40036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T06:33:07.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.686+0000 7fdaea7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559639dbf80 0x555963a55b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:07.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.686+0000 7fdaea7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559639dbf80 0x555963a55b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60344/0 (socket says 172.21.15.44:60344) 2024-04-22T06:33:07.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.686+0000 7fdaea7fc640 1 -- 172.21.15.44:0/3820044488 learned_addr learned my addr 172.21.15.44:0/3820044488 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:33:07.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.686+0000 7fdaeaffd640 1 --2- 172.21.15.44:0/3820044488 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559639a1200 0x5559639db7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:07.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.686+0000 7fdaeb7fe640 1 --2- 172.21.15.44:0/3820044488 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5559639eb080 0x555963a560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:07.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.686+0000 7fdaea7fc640 1 -- 172.21.15.44:0/3820044488 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5559639eb080 msgr2=0x555963a560a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:33:07.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.687+0000 7fdaea7fc640 1 --2- 172.21.15.44:0/3820044488 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5559639eb080 0x555963a560a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.687+0000 7fdaea7fc640 1 -- 172.21.15.44:0/3820044488 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559639a1200 msgr2=0x5559639db7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:33:07.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.687+0000 7fdaea7fc640 1 --2- 172.21.15.44:0/3820044488 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559639a1200 0x5559639db7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.687+0000 7fdaea7fc640 1 -- 172.21.15.44:0/3820044488 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559639fadb0 con 0x5559639dbf80 2024-04-22T06:33:07.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.687+0000 7fdaeaffd640 1 --2- 172.21.15.44:0/3820044488 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559639a1200 0x5559639db7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:33:07.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.687+0000 7fdaeb7fe640 1 --2- 172.21.15.44:0/3820044488 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5559639eb080 0x555963a560a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:33:07.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.687+0000 7fdaea7fc640 1 --2- 172.21.15.44:0/3820044488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559639dbf80 0x555963a55b60 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdae0059220 tx=0x7fdae0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a4bd2dae6d26786 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:07.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.687+0000 7fdac9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T06:33:07.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.687+0000 7fdac9ffb640 1 -- 172.21.15.44:0/3820044488 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae0061ff0 con 0x5559639dbf80 2024-04-22T06:33:07.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.687+0000 7fdac9ffb640 1 -- 172.21.15.44:0/3820044488 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(28..28 src has 1..28) ==== 6821+0+0 (secure 0 0 0) 0x7fdae0058ba0 con 0x5559639dbf80 2024-04-22T06:33:07.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.687+0000 7fdaf41e0740 1 -- 172.21.15.44:0/3820044488 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdadc01b040 con 0x5559639dbf80 2024-04-22T06:33:07.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.687+0000 7fdac9ffb640 1 -- 172.21.15.44:0/3820044488 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fdae0062280 con 0x5559639dbf80 2024-04-22T06:33:07.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.687+0000 7fdaf41e0740 10 client.5343 fetch_fsmap learned FSMap version 19 2024-04-22T06:33:07.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.687+0000 7fdaf41e0740 1 -- 172.21.15.44:0/3820044488 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x5559639c7830 con 0x5559639dbf80 2024-04-22T06:33:07.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.687+0000 7fdac9ffb640 1 -- 172.21.15.44:0/3820044488 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdadc01b040 con 0x5559639dbf80 2024-04-22T06:33:07.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.687+0000 7fdac9ffb640 1 -- 172.21.15.44:0/3820044488 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 19) ==== 4179+0+0 (secure 0 0 0) 0x7fdae0024060 con 0x5559639dbf80 2024-04-22T06:33:07.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.687+0000 7fdaf41e0740 10 client.5343 fetch_fsmap finished waiting for FSMap version 19 2024-04-22T06:33:07.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.687+0000 7fdaf41e0740 10 client.5343 resolve_mds: validated gid 4308 aka daemon mds.d 2024-04-22T06:33:07.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.687+0000 7fdaf41e0740 20 client.5343 populate_metadata read hostname 'smithi044' 2024-04-22T06:33:07.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.687+0000 7fdaf41e0740 1 --2- 172.21.15.44:0/3820044488 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x5559639fa720 0x555963a565e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:07.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.687+0000 7fdaf41e0740 4 client.5343 mds_command: new command op to 4308 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 86.1, "await": true}] 2024-04-22T06:33:07.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.687+0000 7fdaf41e0740 1 -- 172.21.15.44:0/3820044488 --> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 86.1, "await": true}) -- 0x55596390d980 con 0x5559639fa720 2024-04-22T06:33:07.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.688+0000 7fdaeaffd640 1 --2- 172.21.15.44:0/3820044488 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x5559639fa720 0x555963a565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:07.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.688+0000 7fdac9ffb640 10 client.5343 ms_handle_connect on v2:172.21.15.70:6837/2915150040 2024-04-22T06:33:07.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.688+0000 7fdaeaffd640 1 --2- 172.21.15.44:0/3820044488 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x5559639fa720 0x555963a565e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8e6f36781541374 server_cookie=b41b6d5f18e80065 in_seq=0 out_seq=0 2024-04-22T06:33:07.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.688+0000 7fdac9ffb640 1 -- 172.21.15.44:0/3820044488 <== mds.0 v2:172.21.15.70:6837/2915150040 1 ==== command_reply(tid 0: -9 ) ==== 8+0+583 (crc 0 0 0) 0x55596390d980 con 0x5559639fa720 2024-04-22T06:33:07.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.688+0000 7fdac9ffb640 10 client.5343 handle_command_reply: tid=0 2024-04-22T06:33:07.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.689+0000 7fdaf41e0740 1 client.5343 shutdown 2024-04-22T06:33:07.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.689+0000 7fdaf41e0740 1 -- 172.21.15.44:0/3820044488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559639dbf80 msgr2=0x555963a55b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:07.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.689+0000 7fdaf41e0740 1 --2- 172.21.15.44:0/3820044488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559639dbf80 0x555963a55b60 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdae0059220 tx=0x7fdae0000bc0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.689+0000 7fdaf41e0740 1 -- 172.21.15.44:0/3820044488 shutdown_connections 2024-04-22T06:33:07.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.689+0000 7fdaf41e0740 1 --2- 172.21.15.44:0/3820044488 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5559639eb080 0x555963a560a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.689+0000 7fdaf41e0740 1 --2- 172.21.15.44:0/3820044488 >> [v2:172.21.15.70:6837/2915150040,v1:172.21.15.70:6839/2915150040] conn(0x5559639fa720 0x555963a565e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.689+0000 7fdaf41e0740 1 --2- 172.21.15.44:0/3820044488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559639dbf80 0x555963a55b60 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.689+0000 7fdaf41e0740 1 --2- 172.21.15.44:0/3820044488 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5559639a1200 0x5559639db7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:07.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.689+0000 7fdac9ffb640 0 client.5343 ms_handle_reset on v2:172.21.15.70:6837/2915150040 2024-04-22T06:33:07.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.689+0000 7fdaf41e0740 1 -- 172.21.15.44:0/3820044488 >> 172.21.15.44:0/3820044488 conn(0x55596399fc60 msgr2=0x5559639c3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:33:07.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.689+0000 7fdaf41e0740 1 -- 172.21.15.44:0/3820044488 shutdown_connections 2024-04-22T06:33:07.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.689+0000 7fdaf41e0740 1 -- 172.21.15.44:0/3820044488 wait complete. 2024-04-22T06:33:07.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:07.689+0000 7fdaf41e0740 20 client.5343 trim_cache size 0 max 16384 2024-04-22T06:33:07.693 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-22T06:33:07.693 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 19, 2024-04-22T06:33:07.693 INFO:teuthology.orchestra.run.smithi044.stdout: "leader": 4308, 2024-04-22T06:33:07.693 INFO:teuthology.orchestra.run.smithi044.stdout: "set_version": 6, 2024-04-22T06:33:07.693 INFO:teuthology.orchestra.run.smithi044.stdout: "sets": { 2024-04-22T06:33:07.693 INFO:teuthology.orchestra.run.smithi044.stdout: "81f19763": { 2024-04-22T06:33:07.693 INFO:teuthology.orchestra.run.smithi044.stdout: "version": 6, 2024-04-22T06:33:07.693 INFO:teuthology.orchestra.run.smithi044.stdout: "age_ref": 0.0, 2024-04-22T06:33:07.693 INFO:teuthology.orchestra.run.smithi044.stdout: "state": { 2024-04-22T06:33:07.694 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FAILED", 2024-04-22T06:33:07.694 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.0 2024-04-22T06:33:07.694 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-22T06:33:07.694 INFO:teuthology.orchestra.run.smithi044.stdout: "timeout": 60.0, 2024-04-22T06:33:07.694 INFO:teuthology.orchestra.run.smithi044.stdout: "expiration": 86.1, 2024-04-22T06:33:07.694 INFO:teuthology.orchestra.run.smithi044.stdout: "members": { 2024-04-22T06:33:07.694 INFO:teuthology.orchestra.run.smithi044.stdout: "file:/": { 2024-04-22T06:33:07.694 INFO:teuthology.orchestra.run.smithi044.stdout: "excluded": false, 2024-04-22T06:33:07.694 INFO:teuthology.orchestra.run.smithi044.stdout: "state": { 2024-04-22T06:33:07.694 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FAILED", 2024-04-22T06:33:07.694 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.0 2024-04-22T06:33:07.694 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-22T06:33:07.694 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-22T06:33:07.695 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-22T06:33:07.695 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-22T06:33:07.695 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-22T06:33:07.695 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-22T06:33:07.695 DEBUG:teuthology.orchestra.run:got remote process result: 9 2024-04-22T06:33:07.696 INFO:teuthology.orchestra.run.smithi044.stderr:Error EBADF: 2024-04-22T06:33:07.696 ERROR:tasks.quiescer.fs.[cephfs]:Couldn't quiesce root with rc: -9 (EBADF), stdout: { "epoch": 19, "leader": 4308, "set_version": 6, "sets": { "81f19763": { "version": 6, "age_ref": 0.0, "state": { "name": "FAILED", "age": 0.0 }, "timeout": 60.0, "expiration": 86.1, "members": { "file:/": { "excluded": false, "state": { "name": "FAILED", "age": 0.0 } } } } } } 2024-04-22T06:33:07.696 ERROR:tasks.quiescer.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 240, in _run self.do_quiesce(d) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 197, in do_quiesce raise RuntimeError(f"Error quiescing root: {rcinfo}") RuntimeError: Error quiescing root: -9 (EBADF) 2024-04-22T06:33:08.779 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T06:33:08.781 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T06:33:12.973 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~13s 2024-04-22T06:33:12.973 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~13s 2024-04-22T06:33:12.973 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~13s 2024-04-22T06:33:12.973 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~13s 2024-04-22T06:33:12.973 INFO:tasks.daemonwatchdog.daemon_watchdog:quiescer.fs.[cephfs] failed 2024-04-22T06:33:12.973 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2024-04-22T06:33:12.973 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T06:33:12.973 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-22T06:33:12.990 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T06:33:13.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.176+0000 7f556c077640 1 Processor -- start 2024-04-22T06:33:13.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.176+0000 7f556c077640 1 -- start start 2024-04-22T06:33:13.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.177+0000 7f556c077640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5564155240 0x7f556415f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:13.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.177+0000 7f556c077640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f55640cc480 0x7f55640cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:13.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.177+0000 7f556c077640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55640cce30 con 0x7f55640cb320 2024-04-22T06:33:13.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.177+0000 7f556c077640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f556415f610 con 0x7f55640cc480 2024-04-22T06:33:13.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.177+0000 7f556c077640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f556415c060 con 0x7f5564155240 2024-04-22T06:33:13.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.177+0000 7f5569612640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f55640cb320 0x7f55640d0ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:35180/0 (socket says 172.21.15.44:35180) 2024-04-22T06:33:13.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.177+0000 7f5569612640 1 -- 172.21.15.44:0/3118938395 learned_addr learned my addr 172.21.15.44:0/3118938395 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:33:13.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.178+0000 7f5569e13640 1 --2- 172.21.15.44:0/3118938395 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5564155240 0x7f556415f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:13.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.178+0000 7f556a614640 1 --2- 172.21.15.44:0/3118938395 >> v2:172.21.15.70:3300/0 conn(0x7f55640cc480 0x7f55640cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:13.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.178+0000 7f5568e11640 1 -- 172.21.15.44:0/3118938395 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1975648375 0 0) 0x7f55640cce30 con 0x7f55640cb320 2024-04-22T06:33:13.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.178+0000 7f5568e11640 1 -- 172.21.15.44:0/3118938395 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5550003520 con 0x7f55640cb320 2024-04-22T06:33:13.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.178+0000 7f5569e13640 1 -- 172.21.15.44:0/3118938395 >> v2:172.21.15.70:3300/0 conn(0x7f55640cc480 msgr2=0x7f55640cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:33:13.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.178+0000 7f5569e13640 1 --2- 172.21.15.44:0/3118938395 >> v2:172.21.15.70:3300/0 conn(0x7f55640cc480 0x7f55640cc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:13.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.178+0000 7f5569e13640 1 -- 172.21.15.44:0/3118938395 >> v1:172.21.15.44:6789/0 conn(0x7f55640cb320 legacy=0x7f55640d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:13.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.178+0000 7f5569e13640 1 -- 172.21.15.44:0/3118938395 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f556415b200 con 0x7f5564155240 2024-04-22T06:33:13.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.178+0000 7f556a614640 1 --2- 172.21.15.44:0/3118938395 >> v2:172.21.15.70:3300/0 conn(0x7f55640cc480 0x7f55640cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:33:13.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.179+0000 7f5569e13640 1 --2- 172.21.15.44:0/3118938395 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5564155240 0x7f556415f0d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f55600097a0 tx=0x7f556002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=406bfad9e09766f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:13.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.179+0000 7f5568e11640 1 -- 172.21.15.44:0/3118938395 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5560035020 con 0x7f5564155240 2024-04-22T06:33:13.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.179+0000 7f5568e11640 1 -- 172.21.15.44:0/3118938395 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5560031830 con 0x7f5564155240 2024-04-22T06:33:13.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.179+0000 7f5568e11640 1 -- 172.21.15.44:0/3118938395 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5560031c00 con 0x7f5564155240 2024-04-22T06:33:13.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.180+0000 7f556c077640 1 -- 172.21.15.44:0/3118938395 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5564155240 msgr2=0x7f556415f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:13.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.180+0000 7f556c077640 1 --2- 172.21.15.44:0/3118938395 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5564155240 0x7f556415f0d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f55600097a0 tx=0x7f556002b8d0 comp rx=0 tx=0).stop 2024-04-22T06:33:13.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.180+0000 7f556c077640 1 -- 172.21.15.44:0/3118938395 shutdown_connections 2024-04-22T06:33:13.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.180+0000 7f556c077640 1 --2- 172.21.15.44:0/3118938395 >> v2:172.21.15.70:3300/0 conn(0x7f55640cc480 0x7f55640cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:13.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.180+0000 7f556c077640 1 --2- 172.21.15.44:0/3118938395 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5564155240 0x7f556415f0d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:13.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.180+0000 7f556c077640 1 -- 172.21.15.44:0/3118938395 >> 172.21.15.44:0/3118938395 conn(0x7f55640750d0 msgr2=0x7f55640754e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:33:13.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.180+0000 7f556c077640 1 -- 172.21.15.44:0/3118938395 shutdown_connections 2024-04-22T06:33:13.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.180+0000 7f556c077640 1 -- 172.21.15.44:0/3118938395 wait complete. 2024-04-22T06:33:13.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.181+0000 7f556c077640 1 Processor -- start 2024-04-22T06:33:13.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.181+0000 7f556c077640 1 -- start start 2024-04-22T06:33:13.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.182+0000 7f556c077640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55640cb320 0x7f55640b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:13.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.182+0000 7f556c077640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f55640cc480 0x7f55640c8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:13.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.182+0000 7f556c077640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5564155240 0x7f55640c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:13.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.182+0000 7f556c077640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f556415baa0 con 0x7f55640cb320 2024-04-22T06:33:13.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.182+0000 7f556c077640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f556415b7c0 con 0x7f5564155240 2024-04-22T06:33:13.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.182+0000 7f556c077640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f556415bd80 con 0x7f55640cc480 2024-04-22T06:33:13.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.182+0000 7f5569e13640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55640cb320 0x7f55640b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:13.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.182+0000 7f5569e13640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55640cb320 0x7f55640b5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33970/0 (socket says 172.21.15.44:33970) 2024-04-22T06:33:13.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.182+0000 7f5569e13640 1 -- 172.21.15.44:0/4089138944 learned_addr learned my addr 172.21.15.44:0/4089138944 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:33:13.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.182+0000 7f556a614640 1 --2- 172.21.15.44:0/4089138944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5564155240 0x7f55640c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:13.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.182+0000 7f5569612640 1 --2- 172.21.15.44:0/4089138944 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f55640cc480 0x7f55640c8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:13.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.182+0000 7f5569e13640 1 -- 172.21.15.44:0/4089138944 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f55640cc480 msgr2=0x7f55640c8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:13.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.182+0000 7f5569e13640 1 --2- 172.21.15.44:0/4089138944 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f55640cc480 0x7f55640c8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:13.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.182+0000 7f5569e13640 1 -- 172.21.15.44:0/4089138944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5564155240 msgr2=0x7f55640c92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:13.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.182+0000 7f5569e13640 1 --2- 172.21.15.44:0/4089138944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5564155240 0x7f55640c92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:13.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.182+0000 7f5569e13640 1 -- 172.21.15.44:0/4089138944 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55641f1920 con 0x7f55640cb320 2024-04-22T06:33:13.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.182+0000 7f556a614640 1 --2- 172.21.15.44:0/4089138944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5564155240 0x7f55640c92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T06:33:13.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.183+0000 7f5569e13640 1 --2- 172.21.15.44:0/4089138944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55640cb320 0x7f55640b5090 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f55600098c0 tx=0x7f55600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:13.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.183+0000 7f555a7fc640 1 -- 172.21.15.44:0/4089138944 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5560040070 con 0x7f55640cb320 2024-04-22T06:33:13.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.183+0000 7f555a7fc640 1 -- 172.21.15.44:0/4089138944 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f556003e070 con 0x7f55640cb320 2024-04-22T06:33:13.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.183+0000 7f556c077640 1 -- 172.21.15.44:0/4089138944 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5564158cc0 con 0x7f55640cb320 2024-04-22T06:33:13.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.183+0000 7f555a7fc640 1 -- 172.21.15.44:0/4089138944 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5560035040 con 0x7f55640cb320 2024-04-22T06:33:13.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.183+0000 7f556c077640 1 -- 172.21.15.44:0/4089138944 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5564154a10 con 0x7f55640cb320 2024-04-22T06:33:13.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.185+0000 7f556c077640 1 -- 172.21.15.44:0/4089138944 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5564068cc0 con 0x7f55640cb320 2024-04-22T06:33:13.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.185+0000 7f555a7fc640 1 -- 172.21.15.44:0/4089138944 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5560038020 con 0x7f55640cb320 2024-04-22T06:33:13.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.186+0000 7f555a7fc640 1 --2- 172.21.15.44:0/4089138944 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f554806c8f0 0x7f554806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:13.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.186+0000 7f555a7fc640 1 -- 172.21.15.44:0/4089138944 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6821+0+0 (secure 0 0 0) 0x7f55600ad760 con 0x7f55640cb320 2024-04-22T06:33:13.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.186+0000 7f5569612640 1 --2- 172.21.15.44:0/4089138944 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f554806c8f0 0x7f554806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:13.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.187+0000 7f5569612640 1 --2- 172.21.15.44:0/4089138944 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f554806c8f0 0x7f554806edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5564159970 tx=0x7f5554009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:13.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.190+0000 7f555a7fc640 1 -- 172.21.15.44:0/4089138944 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f556007b7c0 con 0x7f55640cb320 2024-04-22T06:33:13.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.520+0000 7f556c077640 1 -- 172.21.15.44:0/4089138944 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f55640c89c0 con 0x7f55640cb320 2024-04-22T06:33:13.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.521+0000 7f555a7fc640 1 -- 172.21.15.44:0/4089138944 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18923 (secure 0 0 0) 0x7f556007b0c0 con 0x7f55640cb320 2024-04-22T06:33:13.520 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:33:13.520 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":28,"fsid":"9b0a34ca-403d-4513-8a66-d3749ba559aa","created":"2024-04-22T06:30:42.334581+0000","modified":"2024-04-22T06:33:07.139734+0000","last_up_change":"2024-04-22T06:30:52.942257+0000","last_in_change":"2024-04-22T06:30:44.467727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T06:30:54.560256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T06:31:04.612387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T06:31:05.312124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-22T06:31:09.025550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b4a6692-53c9-472d-b2f3-1dd29a6679ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6817","nonce":2220268493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6819","nonce":2220268493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6823","nonce":2220268493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6821","nonce":2220268493}]},"public_addr":"172.21.15.44:6817/2220268493","cluster_addr":"172.21.15.44:6819/2220268493","heartbeat_back_addr":"172.21.15.44:6823/2220268493","heartbeat_front_addr":"172.21.15.44:6821/2220268493","state":["exists","up"]},{"osd":1,"uuid":"70a6370a-7c0a-4d64-a0e4-b2e726b4f291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6809","nonce":1278678977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6811","nonce":1278678977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6815","nonce":1278678977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6813","nonce":1278678977}]},"public_addr":"172.21.15.44:6809/1278678977","cluster_addr":"172.21.15.44:6811/1278678977","heartbeat_back_addr":"172.21.15.44:6815/1278678977","heartbeat_front_addr":"172.21.15.44:6813/1278678977","state":["exists","up"]},{"osd":2,"uuid":"f2e51a62-ca91-470d-bd5a-5710c61deee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6801","nonce":2544087903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6803","nonce":2544087903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6807","nonce":2544087903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6805","nonce":2544087903}]},"public_addr":"172.21.15.44:6801/2544087903","cluster_addr":"172.21.15.44:6803/2544087903","heartbeat_back_addr":"172.21.15.44:6807/2544087903","heartbeat_front_addr":"172.21.15.44:6805/2544087903","state":["exists","up"]},{"osd":3,"uuid":"bc514203-9af9-4f1f-b18e-765bdd5de6e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6825","nonce":1692928109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6827","nonce":1692928109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6831","nonce":1692928109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6829","nonce":1692928109}]},"public_addr":"172.21.15.44:6825/1692928109","cluster_addr":"172.21.15.44:6827/1692928109","heartbeat_back_addr":"172.21.15.44:6831/1692928109","heartbeat_front_addr":"172.21.15.44:6829/1692928109","state":["exists","up"]},{"osd":4,"uuid":"357c439a-6e08-4c19-ac18-b68e6c890497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6820","nonce":3802577427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6824","nonce":3802577427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6831","nonce":3802577427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6828","nonce":3802577427}]},"public_addr":"172.21.15.70:6820/3802577427","cluster_addr":"172.21.15.70:6824/3802577427","heartbeat_back_addr":"172.21.15.70:6831/3802577427","heartbeat_front_addr":"172.21.15.70:6828/3802577427","state":["exists","up"]},{"osd":5,"uuid":"952fb88f-d56e-4305-bfa8-d32674332dfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6801","nonce":664757345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6803","nonce":664757345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6807","nonce":664757345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6805","nonce":664757345}]},"public_addr":"172.21.15.70:6801/664757345","cluster_addr":"172.21.15.70:6803/664757345","heartbeat_back_addr":"172.21.15.70:6807/664757345","heartbeat_front_addr":"172.21.15.70:6805/664757345","state":["exists","up"]},{"osd":6,"uuid":"74399b3d-fac2-45ab-9172-2cd95b02450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6817","nonce":1623253482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6821","nonce":1623253482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6827","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6829","nonce":1623253482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6825","nonce":1623253482}]},"public_addr":"172.21.15.70:6817/1623253482","cluster_addr":"172.21.15.70:6821/1623253482","heartbeat_back_addr":"172.21.15.70:6829/1623253482","heartbeat_front_addr":"172.21.15.70:6825/1623253482","state":["exists","up"]},{"osd":7,"uuid":"8f35508d-6fcc-43ff-8ee7-0498a7ab3b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6809","nonce":1551180300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6811","nonce":1551180300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6815","nonce":1551180300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6813","nonce":1551180300}]},"public_addr":"172.21.15.70:6809/1551180300","cluster_addr":"172.21.15.70:6811/1551180300","heartbeat_back_addr":"172.21.15.70:6815/1551180300","heartbeat_front_addr":"172.21.15.70:6813/1551180300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.612502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.653756+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.521169+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.649955+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.640231+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.686638+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.642199+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.700360+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6834/1497302315":"2024-04-23T06:33:07.139686+0000","172.21.15.44:6832/1497302315":"2024-04-23T06:33:07.139686+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T06:33:13.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.524+0000 7f556c077640 1 -- 172.21.15.44:0/4089138944 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f554806c8f0 msgr2=0x7f554806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:13.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.524+0000 7f556c077640 1 --2- 172.21.15.44:0/4089138944 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f554806c8f0 0x7f554806edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5564159970 tx=0x7f5554009040 comp rx=0 tx=0).stop 2024-04-22T06:33:13.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.524+0000 7f556c077640 1 -- 172.21.15.44:0/4089138944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55640cb320 msgr2=0x7f55640b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:13.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.524+0000 7f556c077640 1 --2- 172.21.15.44:0/4089138944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55640cb320 0x7f55640b5090 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f55600098c0 tx=0x7f55600046c0 comp rx=0 tx=0).stop 2024-04-22T06:33:13.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.524+0000 7f556c077640 1 -- 172.21.15.44:0/4089138944 shutdown_connections 2024-04-22T06:33:13.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.524+0000 7f556c077640 1 --2- 172.21.15.44:0/4089138944 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f554806c8f0 0x7f554806edb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:13.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.524+0000 7f556c077640 1 --2- 172.21.15.44:0/4089138944 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f55640cc480 0x7f55640c8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:13.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.524+0000 7f556c077640 1 --2- 172.21.15.44:0/4089138944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5564155240 0x7f55640c92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:13.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.524+0000 7f556c077640 1 --2- 172.21.15.44:0/4089138944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55640cb320 0x7f55640b5090 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:13.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.524+0000 7f556c077640 1 -- 172.21.15.44:0/4089138944 >> 172.21.15.44:0/4089138944 conn(0x7f55640750d0 msgr2=0x7f55640d0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:33:13.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.524+0000 7f556c077640 1 -- 172.21.15.44:0/4089138944 shutdown_connections 2024-04-22T06:33:13.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.524+0000 7f556c077640 1 -- 172.21.15.44:0/4089138944 wait complete. 2024-04-22T06:33:13.538 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T06:30:54.560256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T06:31:04.612387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T06:31:05.312124+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-22T06:31:09.025550+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-22T06:33:13.539 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T06:33:13.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.663+0000 7f03a8d75640 1 Processor -- start 2024-04-22T06:33:13.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.663+0000 7f03a8d75640 1 -- start start 2024-04-22T06:33:13.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.664+0000 7f03a8d75640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f03a41553f0 0x7f03a415bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:13.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.664+0000 7f03a8d75640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f03a406d5a0 0x7f03a40cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:13.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.664+0000 7f03a8d75640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03a40d0fa0 con 0x7f03a4155010 2024-04-22T06:33:13.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.664+0000 7f03a8d75640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f03a40d1280 con 0x7f03a41553f0 2024-04-22T06:33:13.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.664+0000 7f03a8d75640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f03a40d1560 con 0x7f03a406d5a0 2024-04-22T06:33:13.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.664+0000 7f0399d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f03a41553f0 0x7f03a415bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:13.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.664+0000 7f03a259c640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f03a4155010 0x7f03a40c6550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:35192/0 (socket says 172.21.15.44:35192) 2024-04-22T06:33:13.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.664+0000 7f03a259c640 1 -- 172.21.15.44:0/1919431396 learned_addr learned my addr 172.21.15.44:0/1919431396 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:33:13.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.664+0000 7f0399d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f03a41553f0 0x7f03a415bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33976/0 (socket says 172.21.15.44:33976) 2024-04-22T06:33:13.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.664+0000 7f03a2d9d640 1 --2- 172.21.15.44:0/1919431396 >> v2:172.21.15.70:3300/0 conn(0x7f03a406d5a0 0x7f03a40cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:13.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.664+0000 7f0399d9b640 1 -- 172.21.15.44:0/1919431396 >> v2:172.21.15.70:3300/0 conn(0x7f03a406d5a0 msgr2=0x7f03a40cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:33:13.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.664+0000 7f0399d9b640 1 --2- 172.21.15.44:0/1919431396 >> v2:172.21.15.70:3300/0 conn(0x7f03a406d5a0 0x7f03a40cbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:13.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.664+0000 7f0399d9b640 1 -- 172.21.15.44:0/1919431396 >> v1:172.21.15.44:6789/0 conn(0x7f03a4155010 legacy=0x7f03a40c6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:13.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.664+0000 7f0399d9b640 1 -- 172.21.15.44:0/1919431396 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a40c5d50 con 0x7f03a41553f0 2024-04-22T06:33:13.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.665+0000 7f0399d9b640 1 --2- 172.21.15.44:0/1919431396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03a41553f0 0x7f03a415bc00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f03a40c77a0 tx=0x7f038c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeb95d9f5c806f05 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:13.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.666+0000 7f03a1d9b640 1 -- 172.21.15.44:0/1919431396 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c034070 con 0x7f03a41553f0 2024-04-22T06:33:13.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.666+0000 7f03a1d9b640 1 -- 172.21.15.44:0/1919431396 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f038c030610 con 0x7f03a41553f0 2024-04-22T06:33:13.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.666+0000 7f03a1d9b640 1 -- 172.21.15.44:0/1919431396 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c0309e0 con 0x7f03a41553f0 2024-04-22T06:33:13.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.666+0000 7f03a8d75640 1 -- 172.21.15.44:0/1919431396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03a41553f0 msgr2=0x7f03a415bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:13.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.666+0000 7f03a8d75640 1 --2- 172.21.15.44:0/1919431396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03a41553f0 0x7f03a415bc00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f03a40c77a0 tx=0x7f038c02b960 comp rx=0 tx=0).stop 2024-04-22T06:33:13.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.666+0000 7f03a8d75640 1 -- 172.21.15.44:0/1919431396 shutdown_connections 2024-04-22T06:33:13.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.666+0000 7f03a8d75640 1 --2- 172.21.15.44:0/1919431396 >> v2:172.21.15.70:3300/0 conn(0x7f03a406d5a0 0x7f03a40cbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:13.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.666+0000 7f03a8d75640 1 --2- 172.21.15.44:0/1919431396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03a41553f0 0x7f03a415bc00 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:13.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.666+0000 7f03a8d75640 1 -- 172.21.15.44:0/1919431396 >> 172.21.15.44:0/1919431396 conn(0x7f03a40750d0 msgr2=0x7f03a40754e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:33:13.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.666+0000 7f03a8d75640 1 -- 172.21.15.44:0/1919431396 shutdown_connections 2024-04-22T06:33:13.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.666+0000 7f03a8d75640 1 -- 172.21.15.44:0/1919431396 wait complete. 2024-04-22T06:33:13.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.668+0000 7f03a8d75640 1 Processor -- start 2024-04-22T06:33:13.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.668+0000 7f03a8d75640 1 -- start start 2024-04-22T06:33:13.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.670+0000 7f03a8d75640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f03a406d5a0 0x7f03a4072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:13.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.670+0000 7f03a8d75640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03a4155010 0x7f03a4073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:13.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.670+0000 7f03a8d75640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03a41553f0 0x7f03a40cead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:13.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.670+0000 7f03a8d75640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f03a40cb450 con 0x7f03a41553f0 2024-04-22T06:33:13.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.670+0000 7f03a8d75640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f03a40cb2d0 con 0x7f03a4155010 2024-04-22T06:33:13.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.670+0000 7f03a8d75640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f03a40cc6f0 con 0x7f03a406d5a0 2024-04-22T06:33:13.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.670+0000 7f03a2d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03a41553f0 0x7f03a40cead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:13.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.670+0000 7f03a2d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03a41553f0 0x7f03a40cead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33988/0 (socket says 172.21.15.44:33988) 2024-04-22T06:33:13.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.670+0000 7f03a2d9d640 1 -- 172.21.15.44:0/394211259 learned_addr learned my addr 172.21.15.44:0/394211259 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:33:13.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.670+0000 7f0399d9b640 1 --2- 172.21.15.44:0/394211259 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03a4155010 0x7f03a4073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:13.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.670+0000 7f03a259c640 1 --2- 172.21.15.44:0/394211259 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f03a406d5a0 0x7f03a4072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:13.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.670+0000 7f03a2d9d640 1 -- 172.21.15.44:0/394211259 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f03a406d5a0 msgr2=0x7f03a4072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:13.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.670+0000 7f03a2d9d640 1 --2- 172.21.15.44:0/394211259 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f03a406d5a0 0x7f03a4072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:13.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.670+0000 7f03a2d9d640 1 -- 172.21.15.44:0/394211259 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03a4155010 msgr2=0x7f03a4073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:13.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.670+0000 7f03a2d9d640 1 --2- 172.21.15.44:0/394211259 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03a4155010 0x7f03a4073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:13.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.670+0000 7f03a2d9d640 1 -- 172.21.15.44:0/394211259 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a4150c00 con 0x7f03a41553f0 2024-04-22T06:33:13.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.670+0000 7f03a259c640 1 --2- 172.21.15.44:0/394211259 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f03a406d5a0 0x7f03a4072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:33:13.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.670+0000 7f0399d9b640 1 --2- 172.21.15.44:0/394211259 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03a4155010 0x7f03a4073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:33:13.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.670+0000 7f03a2d9d640 1 --2- 172.21.15.44:0/394211259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03a41553f0 0x7f03a40cead0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f039c007e60 tx=0x7f039c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:13.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.671+0000 7f039b7fe640 1 -- 172.21.15.44:0/394211259 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039c014420 con 0x7f03a41553f0 2024-04-22T06:33:13.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.671+0000 7f039b7fe640 1 -- 172.21.15.44:0/394211259 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f039c019070 con 0x7f03a41553f0 2024-04-22T06:33:13.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.671+0000 7f039b7fe640 1 -- 172.21.15.44:0/394211259 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039c014820 con 0x7f03a41553f0 2024-04-22T06:33:13.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.671+0000 7f03a8d75640 1 -- 172.21.15.44:0/394211259 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03a414f9b0 con 0x7f03a41553f0 2024-04-22T06:33:13.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.671+0000 7f03a8d75640 1 -- 172.21.15.44:0/394211259 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03a4150670 con 0x7f03a41553f0 2024-04-22T06:33:13.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.672+0000 7f03a8d75640 1 -- 172.21.15.44:0/394211259 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03a4069310 con 0x7f03a41553f0 2024-04-22T06:33:13.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.673+0000 7f039b7fe640 1 -- 172.21.15.44:0/394211259 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f039c0149c0 con 0x7f03a41553f0 2024-04-22T06:33:13.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.673+0000 7f039b7fe640 1 --2- 172.21.15.44:0/394211259 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f038406c940 0x7f038406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:13.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.673+0000 7f039b7fe640 1 -- 172.21.15.44:0/394211259 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6821+0+0 (secure 0 0 0) 0x7f039c08c6c0 con 0x7f03a41553f0 2024-04-22T06:33:13.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.673+0000 7f03a259c640 1 --2- 172.21.15.44:0/394211259 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f038406c940 0x7f038406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:13.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.674+0000 7f03a259c640 1 --2- 172.21.15.44:0/394211259 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f038406c940 0x7f038406ee00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0394000f60 tx=0x7f0394009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:13.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.678+0000 7f039b7fe640 1 -- 172.21.15.44:0/394211259 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f039c05a800 con 0x7f03a41553f0 2024-04-22T06:33:13.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.886+0000 7f03a8d75640 1 -- 172.21.15.44:0/394211259 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f03a40cb990 con 0x7f03a41553f0 2024-04-22T06:33:13.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.887+0000 7f039b7fe640 1 -- 172.21.15.44:0/394211259 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7f039c05a100 con 0x7f03a41553f0 2024-04-22T06:33:13.885 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-04-22T06:33:13.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.888+0000 7f03a8d75640 1 -- 172.21.15.44:0/394211259 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f038406c940 msgr2=0x7f038406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:13.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.889+0000 7f03a8d75640 1 --2- 172.21.15.44:0/394211259 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f038406c940 0x7f038406ee00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0394000f60 tx=0x7f0394009040 comp rx=0 tx=0).stop 2024-04-22T06:33:13.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.889+0000 7f03a8d75640 1 -- 172.21.15.44:0/394211259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03a41553f0 msgr2=0x7f03a40cead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:13.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.889+0000 7f03a8d75640 1 --2- 172.21.15.44:0/394211259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03a41553f0 0x7f03a40cead0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f039c007e60 tx=0x7f039c00e4b0 comp rx=0 tx=0).stop 2024-04-22T06:33:13.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.889+0000 7f03a8d75640 1 -- 172.21.15.44:0/394211259 shutdown_connections 2024-04-22T06:33:13.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.889+0000 7f03a8d75640 1 --2- 172.21.15.44:0/394211259 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f038406c940 0x7f038406ee00 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:13.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.889+0000 7f03a8d75640 1 --2- 172.21.15.44:0/394211259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03a41553f0 0x7f03a40cead0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:13.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.889+0000 7f03a8d75640 1 --2- 172.21.15.44:0/394211259 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03a4155010 0x7f03a4073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:13.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.889+0000 7f03a8d75640 1 --2- 172.21.15.44:0/394211259 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f03a406d5a0 0x7f03a4072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:13.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.889+0000 7f03a8d75640 1 -- 172.21.15.44:0/394211259 >> 172.21.15.44:0/394211259 conn(0x7f03a40750d0 msgr2=0x7f03a40cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:33:13.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.889+0000 7f03a8d75640 1 -- 172.21.15.44:0/394211259 shutdown_connections 2024-04-22T06:33:13.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:13.889+0000 7f03a8d75640 1 -- 172.21.15.44:0/394211259 wait complete. 2024-04-22T06:33:13.899 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T06:33:14.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.021+0000 7f9de1504640 1 Processor -- start 2024-04-22T06:33:14.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.021+0000 7f9de1504640 1 -- start start 2024-04-22T06:33:14.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.023+0000 7f9de1504640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f9ddc1552e0 0x7f9ddc0cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:14.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.023+0000 7f9de1504640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ddc15ccf0 con 0x7f9ddc06d470 2024-04-22T06:33:14.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.023+0000 7f9de1504640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ddc0c5b30 con 0x7f9ddc154f00 2024-04-22T06:33:14.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.023+0000 7f9de1504640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f9ddc0c6740 con 0x7f9ddc1552e0 2024-04-22T06:33:14.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.023+0000 7f9ddb7fe640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f9ddc06d470 0x7f9ddc0d0f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:35198/0 (socket says 172.21.15.44:35198) 2024-04-22T06:33:14.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.023+0000 7f9ddb7fe640 1 -- 172.21.15.44:0/1066759085 learned_addr learned my addr 172.21.15.44:0/1066759085 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:33:14.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.023+0000 7f9dda7fc640 1 --2- 172.21.15.44:0/1066759085 >> v2:172.21.15.44:3300/0 conn(0x7f9ddc1552e0 0x7f9ddc0cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:14.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.023+0000 7f9dd9ffb640 1 -- 172.21.15.44:0/1066759085 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3307824202 0 0) 0x7f9ddc15ccf0 con 0x7f9ddc06d470 2024-04-22T06:33:14.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.023+0000 7f9dd9ffb640 1 -- 172.21.15.44:0/1066759085 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9dc4003520 con 0x7f9ddc06d470 2024-04-22T06:33:14.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.023+0000 7f9dd9ffb640 1 -- 172.21.15.44:0/1066759085 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2947852405 0 0) 0x7f9ddc0c5b30 con 0x7f9ddc154f00 2024-04-22T06:33:14.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.023+0000 7f9dd9ffb640 1 -- 172.21.15.44:0/1066759085 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ddc15ccf0 con 0x7f9ddc154f00 2024-04-22T06:33:14.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.023+0000 7f9dda7fc640 1 -- 172.21.15.44:0/1066759085 >> v1:172.21.15.70:6789/0 conn(0x7f9ddc154f00 legacy=0x7f9ddc15c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:14.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.023+0000 7f9dda7fc640 1 -- 172.21.15.44:0/1066759085 >> v1:172.21.15.44:6789/0 conn(0x7f9ddc06d470 legacy=0x7f9ddc0d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:14.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.023+0000 7f9dda7fc640 1 -- 172.21.15.44:0/1066759085 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ddc0c5e10 con 0x7f9ddc1552e0 2024-04-22T06:33:14.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.025+0000 7f9dda7fc640 1 --2- 172.21.15.44:0/1066759085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ddc1552e0 0x7f9ddc0cb430 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9dd00097a0 tx=0x7f9dd002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6222419be5ad798 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:14.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.025+0000 7f9dd9ffb640 1 -- 172.21.15.44:0/1066759085 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0035020 con 0x7f9ddc1552e0 2024-04-22T06:33:14.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.025+0000 7f9dd9ffb640 1 -- 172.21.15.44:0/1066759085 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dd0031830 con 0x7f9ddc1552e0 2024-04-22T06:33:14.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.025+0000 7f9dd9ffb640 1 -- 172.21.15.44:0/1066759085 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0031c00 con 0x7f9ddc1552e0 2024-04-22T06:33:14.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.026+0000 7f9de1504640 1 -- 172.21.15.44:0/1066759085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ddc1552e0 msgr2=0x7f9ddc0cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:14.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.026+0000 7f9de1504640 1 --2- 172.21.15.44:0/1066759085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ddc1552e0 0x7f9ddc0cb430 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9dd00097a0 tx=0x7f9dd002b8d0 comp rx=0 tx=0).stop 2024-04-22T06:33:14.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.026+0000 7f9de1504640 1 -- 172.21.15.44:0/1066759085 shutdown_connections 2024-04-22T06:33:14.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.026+0000 7f9de1504640 1 --2- 172.21.15.44:0/1066759085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ddc1552e0 0x7f9ddc0cb430 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:14.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.026+0000 7f9de1504640 1 -- 172.21.15.44:0/1066759085 >> 172.21.15.44:0/1066759085 conn(0x7f9ddc0733c0 msgr2=0x7f9ddc0737d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:33:14.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.026+0000 7f9de1504640 1 -- 172.21.15.44:0/1066759085 shutdown_connections 2024-04-22T06:33:14.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.026+0000 7f9de1504640 1 -- 172.21.15.44:0/1066759085 wait complete. 2024-04-22T06:33:14.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.028+0000 7f9de1504640 1 Processor -- start 2024-04-22T06:33:14.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.028+0000 7f9de1504640 1 -- start start 2024-04-22T06:33:14.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.029+0000 7f9de1504640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ddc06d470 0x7f9ddc1506f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:14.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.029+0000 7f9de1504640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ddc154f00 0x7f9ddc150c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:14.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.029+0000 7f9de1504640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ddc15a220 0x7f9ddc15a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:14.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.029+0000 7f9de1504640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc0c6bd0 con 0x7f9ddc15a220 2024-04-22T06:33:14.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.029+0000 7f9de1504640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc0c63d0 con 0x7f9ddc06d470 2024-04-22T06:33:14.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.029+0000 7f9de1504640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9ddc0c6d50 con 0x7f9ddc154f00 2024-04-22T06:33:14.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.029+0000 7f9ddb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ddc15a220 0x7f9ddc15a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:14.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.029+0000 7f9ddb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ddc15a220 0x7f9ddc15a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33998/0 (socket says 172.21.15.44:33998) 2024-04-22T06:33:14.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.029+0000 7f9ddb7fe640 1 -- 172.21.15.44:0/1748578461 learned_addr learned my addr 172.21.15.44:0/1748578461 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:33:14.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.029+0000 7f9ddaffd640 1 --2- 172.21.15.44:0/1748578461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ddc06d470 0x7f9ddc1506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:14.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.029+0000 7f9dda7fc640 1 --2- 172.21.15.44:0/1748578461 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ddc154f00 0x7f9ddc150c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:14.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.029+0000 7f9ddb7fe640 1 -- 172.21.15.44:0/1748578461 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ddc154f00 msgr2=0x7f9ddc150c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:14.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.029+0000 7f9ddb7fe640 1 --2- 172.21.15.44:0/1748578461 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ddc154f00 0x7f9ddc150c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:14.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.029+0000 7f9ddb7fe640 1 -- 172.21.15.44:0/1748578461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ddc06d470 msgr2=0x7f9ddc1506f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:14.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.029+0000 7f9ddb7fe640 1 --2- 172.21.15.44:0/1748578461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ddc06d470 0x7f9ddc1506f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:14.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.029+0000 7f9ddb7fe640 1 -- 172.21.15.44:0/1748578461 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ddc14fa30 con 0x7f9ddc15a220 2024-04-22T06:33:14.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.030+0000 7f9ddb7fe640 1 --2- 172.21.15.44:0/1748578461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ddc15a220 0x7f9ddc15a6a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9dd400cbd0 tx=0x7f9dd400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:14.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.030+0000 7f9dc37fe640 1 -- 172.21.15.44:0/1748578461 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4017070 con 0x7f9ddc15a220 2024-04-22T06:33:14.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.030+0000 7f9dc37fe640 1 -- 172.21.15.44:0/1748578461 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dd400ace0 con 0x7f9ddc15a220 2024-04-22T06:33:14.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.030+0000 7f9de1504640 1 -- 172.21.15.44:0/1748578461 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ddc14fcf0 con 0x7f9ddc15a220 2024-04-22T06:33:14.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.030+0000 7f9dc37fe640 1 -- 172.21.15.44:0/1748578461 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4012620 con 0x7f9ddc15a220 2024-04-22T06:33:14.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.030+0000 7f9de1504640 1 -- 172.21.15.44:0/1748578461 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ddc1f1670 con 0x7f9ddc15a220 2024-04-22T06:33:14.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.031+0000 7f9de1504640 1 -- 172.21.15.44:0/1748578461 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ddc068cc0 con 0x7f9ddc15a220 2024-04-22T06:33:14.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.032+0000 7f9dc37fe640 1 -- 172.21.15.44:0/1748578461 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9dd401a070 con 0x7f9ddc15a220 2024-04-22T06:33:14.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.032+0000 7f9dc37fe640 1 --2- 172.21.15.44:0/1748578461 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f9dc806c920 0x7f9dc806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:14.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.033+0000 7f9dc37fe640 1 -- 172.21.15.44:0/1748578461 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6821+0+0 (secure 0 0 0) 0x7f9dd408bff0 con 0x7f9ddc15a220 2024-04-22T06:33:14.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.033+0000 7f9ddaffd640 1 --2- 172.21.15.44:0/1748578461 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f9dc806c920 0x7f9dc806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:14.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.033+0000 7f9ddaffd640 1 --2- 172.21.15.44:0/1748578461 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f9dc806c920 0x7f9dc806ede0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc0011e0 tx=0x7f9dcc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:14.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.037+0000 7f9dc37fe640 1 -- 172.21.15.44:0/1748578461 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9dd40598e0 con 0x7f9ddc15a220 2024-04-22T06:33:14.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.267+0000 7f9de1504640 1 -- 172.21.15.44:0/1748578461 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9ddc0caee0 con 0x7f9ddc15a220 2024-04-22T06:33:14.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.267+0000 7f9dc37fe640 1 -- 172.21.15.44:0/1748578461 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7f9dd405dbe0 con 0x7f9ddc15a220 2024-04-22T06:33:14.266 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-04-22T06:33:14.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.269+0000 7f9de1504640 1 -- 172.21.15.44:0/1748578461 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f9dc806c920 msgr2=0x7f9dc806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:14.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.269+0000 7f9de1504640 1 --2- 172.21.15.44:0/1748578461 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f9dc806c920 0x7f9dc806ede0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc0011e0 tx=0x7f9dcc009040 comp rx=0 tx=0).stop 2024-04-22T06:33:14.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.269+0000 7f9de1504640 1 -- 172.21.15.44:0/1748578461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ddc15a220 msgr2=0x7f9ddc15a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:14.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.269+0000 7f9de1504640 1 --2- 172.21.15.44:0/1748578461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ddc15a220 0x7f9ddc15a6a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9dd400cbd0 tx=0x7f9dd400a430 comp rx=0 tx=0).stop 2024-04-22T06:33:14.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.269+0000 7f9de1504640 1 -- 172.21.15.44:0/1748578461 shutdown_connections 2024-04-22T06:33:14.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.269+0000 7f9de1504640 1 --2- 172.21.15.44:0/1748578461 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f9dc806c920 0x7f9dc806ede0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:14.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.269+0000 7f9de1504640 1 --2- 172.21.15.44:0/1748578461 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ddc154f00 0x7f9ddc150c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:14.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.269+0000 7f9de1504640 1 --2- 172.21.15.44:0/1748578461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ddc15a220 0x7f9ddc15a6a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:14.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.269+0000 7f9de1504640 1 --2- 172.21.15.44:0/1748578461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ddc06d470 0x7f9ddc1506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:14.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.269+0000 7f9de1504640 1 -- 172.21.15.44:0/1748578461 >> 172.21.15.44:0/1748578461 conn(0x7f9ddc0733c0 msgr2=0x7f9ddc1529e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:33:14.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.270+0000 7f9de1504640 1 -- 172.21.15.44:0/1748578461 shutdown_connections 2024-04-22T06:33:14.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.270+0000 7f9de1504640 1 -- 172.21.15.44:0/1748578461 wait complete. 2024-04-22T06:33:14.283 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T06:33:14.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.415+0000 7f2fe23bd640 1 Processor -- start 2024-04-22T06:33:14.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.416+0000 7f2fe23bd640 1 -- start start 2024-04-22T06:33:14.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.417+0000 7f2fe23bd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2fdc0c8710 0x7f2fdc151e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:14.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.417+0000 7f2fe23bd640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fdc0c9e70 con 0x7f2fdc0cd970 2024-04-22T06:33:14.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.417+0000 7f2fe23bd640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fdc0ca150 con 0x7f2fdc0cdd50 2024-04-22T06:33:14.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.417+0000 7f2fe23bd640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2fdc072c40 con 0x7f2fdc0c8710 2024-04-22T06:33:14.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.417+0000 7f2fdbfff640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f2fdc0cd970 0x7f2fdc158a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:35214/0 (socket says 172.21.15.44:35214) 2024-04-22T06:33:14.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.417+0000 7f2fdbfff640 1 -- 172.21.15.44:0/2732033251 learned_addr learned my addr 172.21.15.44:0/2732033251 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:33:14.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.417+0000 7f2fe095a640 1 --2- 172.21.15.44:0/2732033251 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2fdc0c8710 0x7f2fdc151e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:14.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.417+0000 7f2fdaffd640 1 -- 172.21.15.44:0/2732033251 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2757326698 0 0) 0x7f2fdc0c9e70 con 0x7f2fdc0cd970 2024-04-22T06:33:14.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.417+0000 7f2fdaffd640 1 -- 172.21.15.44:0/2732033251 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2fc4005730 con 0x7f2fdc0cd970 2024-04-22T06:33:14.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.417+0000 7f2fdaffd640 1 -- 172.21.15.44:0/2732033251 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1069993931 0 0) 0x7f2fdc0ca150 con 0x7f2fdc0cdd50 2024-04-22T06:33:14.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.418+0000 7f2fdaffd640 1 -- 172.21.15.44:0/2732033251 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2fdc0c9e70 con 0x7f2fdc0cdd50 2024-04-22T06:33:14.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.418+0000 7f2fe095a640 1 -- 172.21.15.44:0/2732033251 >> v1:172.21.15.70:6789/0 conn(0x7f2fdc0cdd50 legacy=0x7f2fdc072530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:14.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.418+0000 7f2fe095a640 1 -- 172.21.15.44:0/2732033251 >> v1:172.21.15.44:6789/0 conn(0x7f2fdc0cd970 legacy=0x7f2fdc158a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:14.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.418+0000 7f2fe095a640 1 -- 172.21.15.44:0/2732033251 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fdc0ca430 con 0x7f2fdc0c8710 2024-04-22T06:33:14.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.419+0000 7f2fe095a640 1 --2- 172.21.15.44:0/2732033251 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2fdc0c8710 0x7f2fdc151e50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4009dc0 tx=0x7f2fd402bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=873ff1f3514d9f84 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:14.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.419+0000 7f2fdaffd640 1 -- 172.21.15.44:0/2732033251 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4039070 con 0x7f2fdc0c8710 2024-04-22T06:33:14.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.419+0000 7f2fdaffd640 1 -- 172.21.15.44:0/2732033251 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fd402ccd0 con 0x7f2fdc0c8710 2024-04-22T06:33:14.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.419+0000 7f2fdaffd640 1 -- 172.21.15.44:0/2732033251 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4034640 con 0x7f2fdc0c8710 2024-04-22T06:33:14.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.420+0000 7f2fe23bd640 1 -- 172.21.15.44:0/2732033251 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2fdc0c8710 msgr2=0x7f2fdc151e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:14.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.420+0000 7f2fe23bd640 1 --2- 172.21.15.44:0/2732033251 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2fdc0c8710 0x7f2fdc151e50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4009dc0 tx=0x7f2fd402bf20 comp rx=0 tx=0).stop 2024-04-22T06:33:14.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.420+0000 7f2fe23bd640 1 -- 172.21.15.44:0/2732033251 shutdown_connections 2024-04-22T06:33:14.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.420+0000 7f2fe23bd640 1 --2- 172.21.15.44:0/2732033251 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2fdc0c8710 0x7f2fdc151e50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:14.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.420+0000 7f2fe23bd640 1 -- 172.21.15.44:0/2732033251 >> 172.21.15.44:0/2732033251 conn(0x7f2fdc06e8a0 msgr2=0x7f2fdc06fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:33:14.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.420+0000 7f2fe23bd640 1 -- 172.21.15.44:0/2732033251 shutdown_connections 2024-04-22T06:33:14.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.420+0000 7f2fe23bd640 1 -- 172.21.15.44:0/2732033251 wait complete. 2024-04-22T06:33:14.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.422+0000 7f2fe23bd640 1 Processor -- start 2024-04-22T06:33:14.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.422+0000 7f2fe23bd640 1 -- start start 2024-04-22T06:33:14.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.423+0000 7f2fe23bd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2fdc0c8710 0x7f2fdc14b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:14.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.423+0000 7f2fe23bd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fdc0cd970 0x7f2fdc1f1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:14.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.423+0000 7f2fe23bd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fdc0cdd50 0x7f2fdc159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:14.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.423+0000 7f2fe23bd640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2fdc0ca9f0 con 0x7f2fdc0cd970 2024-04-22T06:33:14.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.423+0000 7f2fe23bd640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2fdc0c8b10 con 0x7f2fdc0cdd50 2024-04-22T06:33:14.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.423+0000 7f2fe23bd640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2fdc0caef0 con 0x7f2fdc0c8710 2024-04-22T06:33:14.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.423+0000 7f2fdb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fdc0cd970 0x7f2fdc1f1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:14.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.423+0000 7f2fdb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fdc0cd970 0x7f2fdc1f1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34000/0 (socket says 172.21.15.44:34000) 2024-04-22T06:33:14.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.423+0000 7f2fdb7fe640 1 -- 172.21.15.44:0/2987836106 learned_addr learned my addr 172.21.15.44:0/2987836106 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:33:14.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.423+0000 7f2fdbfff640 1 --2- 172.21.15.44:0/2987836106 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2fdc0c8710 0x7f2fdc14b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:14.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.423+0000 7f2fe095a640 1 --2- 172.21.15.44:0/2987836106 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fdc0cdd50 0x7f2fdc159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:14.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.424+0000 7f2fdb7fe640 1 -- 172.21.15.44:0/2987836106 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2fdc0c8710 msgr2=0x7f2fdc14b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:14.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.424+0000 7f2fdb7fe640 1 --2- 172.21.15.44:0/2987836106 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2fdc0c8710 0x7f2fdc14b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:14.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.424+0000 7f2fdb7fe640 1 -- 172.21.15.44:0/2987836106 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fdc0cdd50 msgr2=0x7f2fdc159480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:14.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.424+0000 7f2fdb7fe640 1 --2- 172.21.15.44:0/2987836106 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fdc0cdd50 0x7f2fdc159480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:14.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.424+0000 7f2fdb7fe640 1 -- 172.21.15.44:0/2987836106 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fdc14c490 con 0x7f2fdc0cd970 2024-04-22T06:33:14.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.424+0000 7f2fdb7fe640 1 --2- 172.21.15.44:0/2987836106 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fdc0cd970 0x7f2fdc1f1300 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2fd000c480 tx=0x7f2fd000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:14.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.424+0000 7f2fd8ff9640 1 -- 172.21.15.44:0/2987836106 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd0017070 con 0x7f2fdc0cd970 2024-04-22T06:33:14.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.424+0000 7f2fd8ff9640 1 -- 172.21.15.44:0/2987836106 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fd0005480 con 0x7f2fdc0cd970 2024-04-22T06:33:14.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.424+0000 7f2fd8ff9640 1 -- 172.21.15.44:0/2987836106 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd0003830 con 0x7f2fdc0cd970 2024-04-22T06:33:14.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.424+0000 7f2fe23bd640 1 -- 172.21.15.44:0/2987836106 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fdc15b230 con 0x7f2fdc0cd970 2024-04-22T06:33:14.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.424+0000 7f2fe23bd640 1 -- 172.21.15.44:0/2987836106 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fdc15b7f0 con 0x7f2fdc0cd970 2024-04-22T06:33:14.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.425+0000 7f2fe23bd640 1 -- 172.21.15.44:0/2987836106 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fdc072180 con 0x7f2fdc0cd970 2024-04-22T06:33:14.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.426+0000 7f2fd8ff9640 1 -- 172.21.15.44:0/2987836106 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2fd001a070 con 0x7f2fdc0cd970 2024-04-22T06:33:14.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.426+0000 7f2fd8ff9640 1 --2- 172.21.15.44:0/2987836106 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f2fc806c920 0x7f2fc806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:14.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.426+0000 7f2fd8ff9640 1 -- 172.21.15.44:0/2987836106 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6821+0+0 (secure 0 0 0) 0x7f2fd008b220 con 0x7f2fdc0cd970 2024-04-22T06:33:14.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.426+0000 7f2fdbfff640 1 --2- 172.21.15.44:0/2987836106 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f2fc806c920 0x7f2fc806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:14.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.427+0000 7f2fdbfff640 1 --2- 172.21.15.44:0/2987836106 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f2fc806c920 0x7f2fc806ede0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc0011e0 tx=0x7f2fcc007d30 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:14.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.431+0000 7f2fd8ff9640 1 -- 172.21.15.44:0/2987836106 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2fdc072180 con 0x7f2fdc0cd970 2024-04-22T06:33:14.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.682+0000 7f2fe23bd640 1 -- 172.21.15.44:0/2987836106 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2fdc069310 con 0x7f2fdc0cd970 2024-04-22T06:33:14.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.683+0000 7f2fd8ff9640 1 -- 172.21.15.44:0/2987836106 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7f2fd0058c30 con 0x7f2fdc0cd970 2024-04-22T06:33:14.682 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-04-22T06:33:14.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.685+0000 7f2fe23bd640 1 -- 172.21.15.44:0/2987836106 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f2fc806c920 msgr2=0x7f2fc806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:14.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.685+0000 7f2fe23bd640 1 --2- 172.21.15.44:0/2987836106 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f2fc806c920 0x7f2fc806ede0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc0011e0 tx=0x7f2fcc007d30 comp rx=0 tx=0).stop 2024-04-22T06:33:14.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.685+0000 7f2fe23bd640 1 -- 172.21.15.44:0/2987836106 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fdc0cd970 msgr2=0x7f2fdc1f1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:14.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.685+0000 7f2fe23bd640 1 --2- 172.21.15.44:0/2987836106 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fdc0cd970 0x7f2fdc1f1300 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2fd000c480 tx=0x7f2fd000c940 comp rx=0 tx=0).stop 2024-04-22T06:33:14.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.685+0000 7f2fe23bd640 1 -- 172.21.15.44:0/2987836106 shutdown_connections 2024-04-22T06:33:14.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.685+0000 7f2fe23bd640 1 --2- 172.21.15.44:0/2987836106 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f2fc806c920 0x7f2fc806ede0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:14.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.685+0000 7f2fe23bd640 1 --2- 172.21.15.44:0/2987836106 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fdc0cdd50 0x7f2fdc159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:14.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.685+0000 7f2fe23bd640 1 --2- 172.21.15.44:0/2987836106 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fdc0cd970 0x7f2fdc1f1300 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:14.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.685+0000 7f2fe23bd640 1 --2- 172.21.15.44:0/2987836106 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2fdc0c8710 0x7f2fdc14b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:14.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.685+0000 7f2fe23bd640 1 -- 172.21.15.44:0/2987836106 >> 172.21.15.44:0/2987836106 conn(0x7f2fdc06e8a0 msgr2=0x7f2fdc072dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:33:14.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.685+0000 7f2fe23bd640 1 -- 172.21.15.44:0/2987836106 shutdown_connections 2024-04-22T06:33:14.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.685+0000 7f2fe23bd640 1 -- 172.21.15.44:0/2987836106 wait complete. 2024-04-22T06:33:14.695 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-22T06:33:14.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.800+0000 7fd0796e0640 1 Processor -- start 2024-04-22T06:33:14.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.800+0000 7fd0796e0640 1 -- start start 2024-04-22T06:33:14.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.801+0000 7fd0796e0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd07415bc90 0x7fd074070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:14.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.801+0000 7fd0796e0640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd0740711e0 0x7fd074165b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:14.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.801+0000 7fd0796e0640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd07406d4e0 con 0x7fd0741551e0 2024-04-22T06:33:14.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.801+0000 7fd0796e0640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fd074070810 con 0x7fd0740711e0 2024-04-22T06:33:14.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.801+0000 7fd0796e0640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fd074073380 con 0x7fd07415bc90 2024-04-22T06:33:14.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.801+0000 7fd0737fe640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd0740711e0 0x7fd074165b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:14.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.801+0000 7fd0737fe640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd0740711e0 0x7fd074165b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34002/0 (socket says 172.21.15.44:34002) 2024-04-22T06:33:14.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.801+0000 7fd0737fe640 1 -- 172.21.15.44:0/3593498210 learned_addr learned my addr 172.21.15.44:0/3593498210 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:33:14.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.801+0000 7fd0727fc640 1 --2- 172.21.15.44:0/3593498210 >> v2:172.21.15.70:3300/0 conn(0x7fd07415bc90 0x7fd074070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:14.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.802+0000 7fd0737fe640 1 -- 172.21.15.44:0/3593498210 >> v2:172.21.15.70:3300/0 conn(0x7fd07415bc90 msgr2=0x7fd074070ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:33:14.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.802+0000 7fd0737fe640 1 --2- 172.21.15.44:0/3593498210 >> v2:172.21.15.70:3300/0 conn(0x7fd07415bc90 0x7fd074070ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:14.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.802+0000 7fd071ffb640 1 -- 172.21.15.44:0/3593498210 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3672903132 0 0) 0x7fd07406d4e0 con 0x7fd0741551e0 2024-04-22T06:33:14.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.802+0000 7fd0737fe640 1 -- 172.21.15.44:0/3593498210 >> v1:172.21.15.44:6789/0 conn(0x7fd0741551e0 legacy=0x7fd07406b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:14.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.802+0000 7fd0737fe640 1 -- 172.21.15.44:0/3593498210 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd074073660 con 0x7fd0740711e0 2024-04-22T06:33:14.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.803+0000 7fd0737fe640 1 --2- 172.21.15.44:0/3593498210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0740711e0 0x7fd074165b90 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd074050d80 tx=0x7fd06c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3d4c6e29d4acebf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:14.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.803+0000 7fd071ffb640 1 -- 172.21.15.44:0/3593498210 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c02cb20 con 0x7fd0740711e0 2024-04-22T06:33:14.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.803+0000 7fd071ffb640 1 -- 172.21.15.44:0/3593498210 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd06c02ccc0 con 0x7fd0740711e0 2024-04-22T06:33:14.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.803+0000 7fd071ffb640 1 -- 172.21.15.44:0/3593498210 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c034610 con 0x7fd0740711e0 2024-04-22T06:33:14.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.804+0000 7fd0796e0640 1 -- 172.21.15.44:0/3593498210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0740711e0 msgr2=0x7fd074165b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:14.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.804+0000 7fd0796e0640 1 --2- 172.21.15.44:0/3593498210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0740711e0 0x7fd074165b90 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd074050d80 tx=0x7fd06c02bf80 comp rx=0 tx=0).stop 2024-04-22T06:33:14.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.804+0000 7fd0796e0640 1 -- 172.21.15.44:0/3593498210 shutdown_connections 2024-04-22T06:33:14.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.804+0000 7fd0796e0640 1 --2- 172.21.15.44:0/3593498210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0740711e0 0x7fd074165b90 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:14.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.804+0000 7fd0796e0640 1 --2- 172.21.15.44:0/3593498210 >> v2:172.21.15.70:3300/0 conn(0x7fd07415bc90 0x7fd074070ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:14.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.804+0000 7fd0796e0640 1 -- 172.21.15.44:0/3593498210 >> 172.21.15.44:0/3593498210 conn(0x7fd0740ce700 msgr2=0x7fd0740ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:33:14.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.804+0000 7fd0796e0640 1 -- 172.21.15.44:0/3593498210 shutdown_connections 2024-04-22T06:33:14.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.804+0000 7fd0796e0640 1 -- 172.21.15.44:0/3593498210 wait complete. 2024-04-22T06:33:14.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.806+0000 7fd0796e0640 1 Processor -- start 2024-04-22T06:33:14.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.806+0000 7fd0796e0640 1 -- start start 2024-04-22T06:33:14.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.807+0000 7fd0796e0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0740711e0 0x7fd0740c8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:14.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.807+0000 7fd0796e0640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0741551e0 0x7fd0740c5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:14.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.807+0000 7fd0796e0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd07415bc90 0x7fd0740cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:14.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.807+0000 7fd072ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0740711e0 0x7fd0740c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:14.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.807+0000 7fd0796e0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd0740cf420 con 0x7fd0740711e0 2024-04-22T06:33:14.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.807+0000 7fd0796e0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd074073c20 con 0x7fd07415bc90 2024-04-22T06:33:14.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.807+0000 7fd0796e0640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd0740715f0 con 0x7fd0741551e0 2024-04-22T06:33:14.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.807+0000 7fd072ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0740711e0 0x7fd0740c8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34018/0 (socket says 172.21.15.44:34018) 2024-04-22T06:33:14.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.807+0000 7fd072ffd640 1 -- 172.21.15.44:0/993569022 learned_addr learned my addr 172.21.15.44:0/993569022 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:33:14.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.807+0000 7fd0727fc640 1 --2- 172.21.15.44:0/993569022 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0741551e0 0x7fd0740c5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:14.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.808+0000 7fd0737fe640 1 --2- 172.21.15.44:0/993569022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd07415bc90 0x7fd0740cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:14.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.808+0000 7fd072ffd640 1 -- 172.21.15.44:0/993569022 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0741551e0 msgr2=0x7fd0740c5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:14.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.808+0000 7fd072ffd640 1 --2- 172.21.15.44:0/993569022 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0741551e0 0x7fd0740c5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:14.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.808+0000 7fd072ffd640 1 -- 172.21.15.44:0/993569022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd07415bc90 msgr2=0x7fd0740cb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:14.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.808+0000 7fd072ffd640 1 --2- 172.21.15.44:0/993569022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd07415bc90 0x7fd0740cb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:14.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.808+0000 7fd072ffd640 1 -- 172.21.15.44:0/993569022 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0741edbb0 con 0x7fd0740711e0 2024-04-22T06:33:14.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.808+0000 7fd072ffd640 1 --2- 172.21.15.44:0/993569022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0740711e0 0x7fd0740c8430 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd06400c4b0 tx=0x7fd06400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:14.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.808+0000 7fd05b7fe640 1 -- 172.21.15.44:0/993569022 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd064017070 con 0x7fd0740711e0 2024-04-22T06:33:14.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.808+0000 7fd05b7fe640 1 -- 172.21.15.44:0/993569022 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd064005480 con 0x7fd0740711e0 2024-04-22T06:33:14.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.808+0000 7fd0796e0640 1 -- 172.21.15.44:0/993569022 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0741ede90 con 0x7fd0740711e0 2024-04-22T06:33:14.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.808+0000 7fd05b7fe640 1 -- 172.21.15.44:0/993569022 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd064003830 con 0x7fd0740711e0 2024-04-22T06:33:14.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.808+0000 7fd0796e0640 1 -- 172.21.15.44:0/993569022 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0740c7c30 con 0x7fd0740711e0 2024-04-22T06:33:14.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.809+0000 7fd0796e0640 1 -- 172.21.15.44:0/993569022 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd074069310 con 0x7fd0740711e0 2024-04-22T06:33:14.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.810+0000 7fd05b7fe640 1 -- 172.21.15.44:0/993569022 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd06401a070 con 0x7fd0740711e0 2024-04-22T06:33:14.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.811+0000 7fd05b7fe640 1 --2- 172.21.15.44:0/993569022 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd06006c8f0 0x7fd06006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:14.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.811+0000 7fd05b7fe640 1 -- 172.21.15.44:0/993569022 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6821+0+0 (secure 0 0 0) 0x7fd06408b360 con 0x7fd0740711e0 2024-04-22T06:33:14.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.811+0000 7fd0727fc640 1 --2- 172.21.15.44:0/993569022 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd06006c8f0 0x7fd06006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:14.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.812+0000 7fd0727fc640 1 --2- 172.21.15.44:0/993569022 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd06006c8f0 0x7fd06006edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd074151540 tx=0x7fd068009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:14.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:14.815+0000 7fd05b7fe640 1 -- 172.21.15.44:0/993569022 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd064058c50 con 0x7fd0740711e0 2024-04-22T06:33:14.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.000+0000 7fd0796e0640 1 -- 172.21.15.44:0/993569022 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd0740c8ac0 con 0x7fd0740711e0 2024-04-22T06:33:14.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.000+0000 7fd05b7fe640 1 -- 172.21.15.44:0/993569022 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7fd06405cf50 con 0x7fd0740711e0 2024-04-22T06:33:14.999 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-04-22T06:33:15.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.003+0000 7fd0796e0640 1 -- 172.21.15.44:0/993569022 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd06006c8f0 msgr2=0x7fd06006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:15.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.003+0000 7fd0796e0640 1 --2- 172.21.15.44:0/993569022 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd06006c8f0 0x7fd06006edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd074151540 tx=0x7fd068009190 comp rx=0 tx=0).stop 2024-04-22T06:33:15.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.003+0000 7fd0796e0640 1 -- 172.21.15.44:0/993569022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0740711e0 msgr2=0x7fd0740c8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:15.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.003+0000 7fd0796e0640 1 --2- 172.21.15.44:0/993569022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0740711e0 0x7fd0740c8430 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd06400c4b0 tx=0x7fd06400c970 comp rx=0 tx=0).stop 2024-04-22T06:33:15.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.003+0000 7fd0796e0640 1 -- 172.21.15.44:0/993569022 shutdown_connections 2024-04-22T06:33:15.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.003+0000 7fd0796e0640 1 --2- 172.21.15.44:0/993569022 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fd06006c8f0 0x7fd06006edb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:15.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.003+0000 7fd0796e0640 1 --2- 172.21.15.44:0/993569022 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd0741551e0 0x7fd0740c5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:15.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.003+0000 7fd0796e0640 1 --2- 172.21.15.44:0/993569022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd07415bc90 0x7fd0740cb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:15.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.003+0000 7fd0796e0640 1 --2- 172.21.15.44:0/993569022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0740711e0 0x7fd0740c8430 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:15.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.003+0000 7fd0796e0640 1 -- 172.21.15.44:0/993569022 >> 172.21.15.44:0/993569022 conn(0x7fd0740ce700 msgr2=0x7fd074152a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:33:15.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.003+0000 7fd0796e0640 1 -- 172.21.15.44:0/993569022 shutdown_connections 2024-04-22T06:33:15.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.003+0000 7fd0796e0640 1 -- 172.21.15.44:0/993569022 wait complete. 2024-04-22T06:33:15.017 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-22T06:33:15.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.127+0000 7f8ec2515640 1 Processor -- start 2024-04-22T06:33:15.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.127+0000 7f8ec2515640 1 -- start start 2024-04-22T06:33:15.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.128+0000 7f8ec2515640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ebc154f00 0x7f8ebc0cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:15.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.128+0000 7f8ec2515640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ebc15ccf0 con 0x7f8ebc1552e0 2024-04-22T06:33:15.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.128+0000 7f8ec2515640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ebc0c5b30 con 0x7f8ebc06d470 2024-04-22T06:33:15.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.128+0000 7f8ec2515640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8ebc0c6740 con 0x7f8ebc154f00 2024-04-22T06:33:15.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.128+0000 7f8ebb7fe640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f8ebc1552e0 0x7f8ebc15c5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:35240/0 (socket says 172.21.15.44:35240) 2024-04-22T06:33:15.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.128+0000 7f8ebb7fe640 1 -- 172.21.15.44:0/2132356273 learned_addr learned my addr 172.21.15.44:0/2132356273 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:33:15.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.128+0000 7f8ebbfff640 1 --2- 172.21.15.44:0/2132356273 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ebc154f00 0x7f8ebc0cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:15.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.128+0000 7f8ebaffd640 1 -- 172.21.15.44:0/2132356273 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2139282803 0 0) 0x7f8ebc15ccf0 con 0x7f8ebc1552e0 2024-04-22T06:33:15.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.129+0000 7f8ebaffd640 1 -- 172.21.15.44:0/2132356273 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ea4003520 con 0x7f8ebc1552e0 2024-04-22T06:33:15.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.129+0000 7f8ebbfff640 1 -- 172.21.15.44:0/2132356273 >> v1:172.21.15.70:6789/0 conn(0x7f8ebc06d470 legacy=0x7f8ebc0d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:15.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.129+0000 7f8ebbfff640 1 -- 172.21.15.44:0/2132356273 >> v1:172.21.15.44:6789/0 conn(0x7f8ebc1552e0 legacy=0x7f8ebc15c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:15.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.129+0000 7f8ebbfff640 1 -- 172.21.15.44:0/2132356273 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ebc0c5e10 con 0x7f8ebc154f00 2024-04-22T06:33:15.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.131+0000 7f8ebbfff640 1 --2- 172.21.15.44:0/2132356273 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ebc154f00 0x7f8ebc0cb430 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8eac009750 tx=0x7f8eac02b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df8e62587b7f2ce2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:15.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.131+0000 7f8ebaffd640 1 -- 172.21.15.44:0/2132356273 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac037070 con 0x7f8ebc154f00 2024-04-22T06:33:15.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.131+0000 7f8ebaffd640 1 -- 172.21.15.44:0/2132356273 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eac030710 con 0x7f8ebc154f00 2024-04-22T06:33:15.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.131+0000 7f8ebaffd640 1 -- 172.21.15.44:0/2132356273 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac030ac0 con 0x7f8ebc154f00 2024-04-22T06:33:15.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.132+0000 7f8ec2515640 1 -- 172.21.15.44:0/2132356273 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ebc154f00 msgr2=0x7f8ebc0cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:15.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.132+0000 7f8ec2515640 1 --2- 172.21.15.44:0/2132356273 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ebc154f00 0x7f8ebc0cb430 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8eac009750 tx=0x7f8eac02b730 comp rx=0 tx=0).stop 2024-04-22T06:33:15.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.132+0000 7f8ec2515640 1 -- 172.21.15.44:0/2132356273 shutdown_connections 2024-04-22T06:33:15.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.132+0000 7f8ec2515640 1 --2- 172.21.15.44:0/2132356273 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ebc154f00 0x7f8ebc0cb430 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:15.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.132+0000 7f8ec2515640 1 -- 172.21.15.44:0/2132356273 >> 172.21.15.44:0/2132356273 conn(0x7f8ebc0733c0 msgr2=0x7f8ebc0737d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:33:15.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.132+0000 7f8ec2515640 1 -- 172.21.15.44:0/2132356273 shutdown_connections 2024-04-22T06:33:15.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.132+0000 7f8ec2515640 1 -- 172.21.15.44:0/2132356273 wait complete. 2024-04-22T06:33:15.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.134+0000 7f8ec2515640 1 Processor -- start 2024-04-22T06:33:15.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.134+0000 7f8ec2515640 1 -- start start 2024-04-22T06:33:15.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.135+0000 7f8ec2515640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ebc06d470 0x7f8ebc1f14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:15.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.135+0000 7f8ec2515640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ebc154f00 0x7f8ebc0ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:15.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.135+0000 7f8ec2515640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ebc1552e0 0x7f8ebc159ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:15.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.135+0000 7f8ec2515640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8ebc0c6bd0 con 0x7f8ebc06d470 2024-04-22T06:33:15.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.135+0000 7f8ec2515640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8ebc0c63d0 con 0x7f8ebc1552e0 2024-04-22T06:33:15.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.135+0000 7f8ec2515640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8ebc0c6d50 con 0x7f8ebc154f00 2024-04-22T06:33:15.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.135+0000 7f8ebbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ebc06d470 0x7f8ebc1f14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:15.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.135+0000 7f8ebbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ebc06d470 0x7f8ebc1f14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34028/0 (socket says 172.21.15.44:34028) 2024-04-22T06:33:15.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.135+0000 7f8ebbfff640 1 -- 172.21.15.44:0/3018348348 learned_addr learned my addr 172.21.15.44:0/3018348348 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:33:15.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.135+0000 7f8ebb7fe640 1 --2- 172.21.15.44:0/3018348348 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ebc154f00 0x7f8ebc0ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:15.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.135+0000 7f8ec0ab2640 1 --2- 172.21.15.44:0/3018348348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ebc1552e0 0x7f8ebc159ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:15.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.136+0000 7f8ebbfff640 1 -- 172.21.15.44:0/3018348348 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ebc154f00 msgr2=0x7f8ebc0ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:15.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.136+0000 7f8ebbfff640 1 --2- 172.21.15.44:0/3018348348 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ebc154f00 0x7f8ebc0ccef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:15.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.136+0000 7f8ebbfff640 1 -- 172.21.15.44:0/3018348348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ebc1552e0 msgr2=0x7f8ebc159ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:15.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.136+0000 7f8ebbfff640 1 --2- 172.21.15.44:0/3018348348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ebc1552e0 0x7f8ebc159ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:15.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.136+0000 7f8ebbfff640 1 -- 172.21.15.44:0/3018348348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ebc15acb0 con 0x7f8ebc06d470 2024-04-22T06:33:15.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.136+0000 7f8ebbfff640 1 --2- 172.21.15.44:0/3018348348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ebc06d470 0x7f8ebc1f14a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8eac035040 tx=0x7f8eac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:15.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.136+0000 7f8eb8ff9640 1 -- 172.21.15.44:0/3018348348 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac041020 con 0x7f8ebc06d470 2024-04-22T06:33:15.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.136+0000 7f8eb8ff9640 1 -- 172.21.15.44:0/3018348348 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eac003610 con 0x7f8ebc06d470 2024-04-22T06:33:15.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.136+0000 7f8ec2515640 1 -- 172.21.15.44:0/3018348348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ebc15a9d0 con 0x7f8ebc06d470 2024-04-22T06:33:15.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.136+0000 7f8ec2515640 1 -- 172.21.15.44:0/3018348348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ebc1592e0 con 0x7f8ebc06d470 2024-04-22T06:33:15.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.136+0000 7f8eb8ff9640 1 -- 172.21.15.44:0/3018348348 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac037040 con 0x7f8ebc06d470 2024-04-22T06:33:15.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.138+0000 7f8ec2515640 1 -- 172.21.15.44:0/3018348348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ebc068cc0 con 0x7f8ebc06d470 2024-04-22T06:33:15.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.138+0000 7f8eb8ff9640 1 -- 172.21.15.44:0/3018348348 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8eac044020 con 0x7f8ebc06d470 2024-04-22T06:33:15.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.138+0000 7f8eb8ff9640 1 --2- 172.21.15.44:0/3018348348 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f8ea806c8f0 0x7f8ea806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:15.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.139+0000 7f8eb8ff9640 1 -- 172.21.15.44:0/3018348348 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6821+0+0 (secure 0 0 0) 0x7f8eac0adad0 con 0x7f8ebc06d470 2024-04-22T06:33:15.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.139+0000 7f8ebb7fe640 1 --2- 172.21.15.44:0/3018348348 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f8ea806c8f0 0x7f8ea806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:15.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.140+0000 7f8ebb7fe640 1 --2- 172.21.15.44:0/3018348348 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f8ea806c8f0 0x7f8ea806edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc154730 tx=0x7f8eb0007d30 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:15.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.143+0000 7f8eb8ff9640 1 -- 172.21.15.44:0/3018348348 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8eac04b020 con 0x7f8ebc06d470 2024-04-22T06:33:15.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.549+0000 7f8ec2515640 1 -- 172.21.15.44:0/3018348348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8ebc154480 con 0x7f8ebc06d470 2024-04-22T06:33:15.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.550+0000 7f8eb8ff9640 1 -- 172.21.15.44:0/3018348348 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v28) ==== 80+0+122 (secure 0 0 0) 0x7f8eac07b450 con 0x7f8ebc06d470 2024-04-22T06:33:15.549 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6834/1497302315 2024-04-23T06:33:07.139686+0000 2024-04-22T06:33:15.549 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6832/1497302315 2024-04-23T06:33:07.139686+0000 2024-04-22T06:33:15.549 INFO:teuthology.orchestra.run.smithi044.stderr:listed 2 entries 2024-04-22T06:33:15.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.552+0000 7f8ec2515640 1 -- 172.21.15.44:0/3018348348 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f8ea806c8f0 msgr2=0x7f8ea806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:15.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.552+0000 7f8ec2515640 1 --2- 172.21.15.44:0/3018348348 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f8ea806c8f0 0x7f8ea806edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc154730 tx=0x7f8eb0007d30 comp rx=0 tx=0).stop 2024-04-22T06:33:15.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.552+0000 7f8ec2515640 1 -- 172.21.15.44:0/3018348348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ebc06d470 msgr2=0x7f8ebc1f14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:15.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.552+0000 7f8ec2515640 1 --2- 172.21.15.44:0/3018348348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ebc06d470 0x7f8ebc1f14a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8eac035040 tx=0x7f8eac0046c0 comp rx=0 tx=0).stop 2024-04-22T06:33:15.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.552+0000 7f8ec2515640 1 -- 172.21.15.44:0/3018348348 shutdown_connections 2024-04-22T06:33:15.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.552+0000 7f8ec2515640 1 --2- 172.21.15.44:0/3018348348 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f8ea806c8f0 0x7f8ea806edb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:15.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.552+0000 7f8ec2515640 1 --2- 172.21.15.44:0/3018348348 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ebc154f00 0x7f8ebc0ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:15.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.552+0000 7f8ec2515640 1 --2- 172.21.15.44:0/3018348348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ebc1552e0 0x7f8ebc159ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:15.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.552+0000 7f8ec2515640 1 --2- 172.21.15.44:0/3018348348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ebc06d470 0x7f8ebc1f14a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:15.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.552+0000 7f8ec2515640 1 -- 172.21.15.44:0/3018348348 >> 172.21.15.44:0/3018348348 conn(0x7f8ebc0733c0 msgr2=0x7f8ebc14f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:33:15.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.553+0000 7f8ec2515640 1 -- 172.21.15.44:0/3018348348 shutdown_connections 2024-04-22T06:33:15.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.553+0000 7f8ec2515640 1 -- 172.21.15.44:0/3018348348 wait complete. 2024-04-22T06:33:15.566 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T06:33:15.566 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-22T06:33:15.581 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T06:33:15.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.753+0000 7fbaef5bb640 1 Processor -- start 2024-04-22T06:33:15.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.754+0000 7fbaef5bb640 1 -- start start 2024-04-22T06:33:15.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.754+0000 7fbaef5bb640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbae8155420 0x7fbae80ca230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:15.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.754+0000 7fbaef5bb640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbae80c9c00 0x7fbae806b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:15.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.754+0000 7fbaef5bb640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbae8073e30 con 0x7fbae815bea0 2024-04-22T06:33:15.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.755+0000 7fbaef5bb640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fbae80740d0 con 0x7fbae80c9c00 2024-04-22T06:33:15.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.755+0000 7fbaef5bb640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fbae815b140 con 0x7fbae8155420 2024-04-22T06:33:15.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.755+0000 7fbaedb58640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbae80c9c00 0x7fbae806b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:15.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.755+0000 7fbaed357640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbae8155420 0x7fbae80ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:15.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.755+0000 7fbaed357640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbae8155420 0x7fbae80ca230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.44:33552/0 (socket says 172.21.15.44:33552) 2024-04-22T06:33:15.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.755+0000 7fbaedb58640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbae80c9c00 0x7fbae806b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:35328/0 (socket says 172.21.15.44:35328) 2024-04-22T06:33:15.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.755+0000 7fbaed357640 1 -- 172.21.15.44:0/121926380 learned_addr learned my addr 172.21.15.44:0/121926380 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:33:15.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.755+0000 7fbaecb56640 1 --1- 172.21.15.44:0/121926380 >> v1:172.21.15.70:6789/0 conn(0x7fbae815bea0 0x7fbae815f0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.44:32880/0 (socket says 172.21.15.44:32880) 2024-04-22T06:33:15.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.755+0000 7fbaedb58640 1 -- 172.21.15.44:0/121926380 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbae8155420 msgr2=0x7fbae80ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:33:15.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.755+0000 7fbaedb58640 1 --2- 172.21.15.44:0/121926380 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbae8155420 0x7fbae80ca230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:15.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.755+0000 7fbaedb58640 1 -- 172.21.15.44:0/121926380 >> v1:172.21.15.70:6789/0 conn(0x7fbae815bea0 legacy=0x7fbae815f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:15.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.755+0000 7fbaedb58640 1 -- 172.21.15.44:0/121926380 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae80ca770 con 0x7fbae80c9c00 2024-04-22T06:33:15.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.755+0000 7fbadffff640 1 -- 172.21.15.44:0/121926380 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3515949880 0 0) 0x7fbae8073e30 con 0x7fbae815bea0 2024-04-22T06:33:15.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.756+0000 7fbaedb58640 1 --2- 172.21.15.44:0/121926380 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbae80c9c00 0x7fbae806b930 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbae0009d90 tx=0x7fbae002bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=435f19b923a8662e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:15.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.757+0000 7fbadffff640 1 -- 172.21.15.44:0/121926380 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae0039070 con 0x7fbae80c9c00 2024-04-22T06:33:15.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.757+0000 7fbadffff640 1 -- 172.21.15.44:0/121926380 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbae002cc80 con 0x7fbae80c9c00 2024-04-22T06:33:15.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.757+0000 7fbadffff640 1 -- 172.21.15.44:0/121926380 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae0034660 con 0x7fbae80c9c00 2024-04-22T06:33:15.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.757+0000 7fbaef5bb640 1 -- 172.21.15.44:0/121926380 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbae80c9c00 msgr2=0x7fbae806b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:15.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.757+0000 7fbaef5bb640 1 --2- 172.21.15.44:0/121926380 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbae80c9c00 0x7fbae806b930 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbae0009d90 tx=0x7fbae002bed0 comp rx=0 tx=0).stop 2024-04-22T06:33:15.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.757+0000 7fbaef5bb640 1 -- 172.21.15.44:0/121926380 shutdown_connections 2024-04-22T06:33:15.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.757+0000 7fbaef5bb640 1 --2- 172.21.15.44:0/121926380 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbae80c9c00 0x7fbae806b930 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:15.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.757+0000 7fbaef5bb640 1 --2- 172.21.15.44:0/121926380 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbae8155420 0x7fbae80ca230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:15.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.757+0000 7fbaef5bb640 1 -- 172.21.15.44:0/121926380 >> 172.21.15.44:0/121926380 conn(0x7fbae8074cb0 msgr2=0x7fbae8075090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:33:15.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.758+0000 7fbaef5bb640 1 -- 172.21.15.44:0/121926380 shutdown_connections 2024-04-22T06:33:15.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.758+0000 7fbaef5bb640 1 -- 172.21.15.44:0/121926380 wait complete. 2024-04-22T06:33:15.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.759+0000 7fbaef5bb640 1 Processor -- start 2024-04-22T06:33:15.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.759+0000 7fbaef5bb640 1 -- start start 2024-04-22T06:33:15.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.760+0000 7fbaef5bb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbae80c9c00 0x7fbae81f1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:15.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.760+0000 7fbaef5bb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbae8155420 0x7fbae80c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:15.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.761+0000 7fbaef5bb640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbae815bea0 0x7fbae8153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:15.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.761+0000 7fbaecb56640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbae8155420 0x7fbae80c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:15.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.761+0000 7fbaecb56640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbae8155420 0x7fbae80c79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34038/0 (socket says 172.21.15.44:34038) 2024-04-22T06:33:15.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.761+0000 7fbaef5bb640 1 -- 172.21.15.44:0/285107267 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbae815b6a0 con 0x7fbae8155420 2024-04-22T06:33:15.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.761+0000 7fbaecb56640 1 -- 172.21.15.44:0/285107267 learned_addr learned my addr 172.21.15.44:0/285107267 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:33:15.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.761+0000 7fbaef5bb640 1 -- 172.21.15.44:0/285107267 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fbae8155800 con 0x7fbae80c9c00 2024-04-22T06:33:15.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.761+0000 7fbaef5bb640 1 -- 172.21.15.44:0/285107267 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fbae8073af0 con 0x7fbae815bea0 2024-04-22T06:33:15.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.761+0000 7fbaed357640 1 --2- 172.21.15.44:0/285107267 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbae80c9c00 0x7fbae81f1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:15.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.761+0000 7fbaedb58640 1 --2- 172.21.15.44:0/285107267 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbae815bea0 0x7fbae8153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:15.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.761+0000 7fbaecb56640 1 -- 172.21.15.44:0/285107267 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbae815bea0 msgr2=0x7fbae8153300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:15.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.761+0000 7fbaecb56640 1 --2- 172.21.15.44:0/285107267 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbae815bea0 0x7fbae8153300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:15.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.761+0000 7fbaecb56640 1 -- 172.21.15.44:0/285107267 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbae80c9c00 msgr2=0x7fbae81f1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:15.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.761+0000 7fbaecb56640 1 --2- 172.21.15.44:0/285107267 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbae80c9c00 0x7fbae81f1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:15.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.761+0000 7fbaecb56640 1 -- 172.21.15.44:0/285107267 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae81f2060 con 0x7fbae8155420 2024-04-22T06:33:15.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.761+0000 7fbaecb56640 1 --2- 172.21.15.44:0/285107267 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbae8155420 0x7fbae80c79c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbad800c460 tx=0x7fbad800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:15.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.762+0000 7fbaddffb640 1 -- 172.21.15.44:0/285107267 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8018070 con 0x7fbae8155420 2024-04-22T06:33:15.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.762+0000 7fbaddffb640 1 -- 172.21.15.44:0/285107267 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbad8007500 con 0x7fbae8155420 2024-04-22T06:33:15.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.762+0000 7fbaef5bb640 1 -- 172.21.15.44:0/285107267 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbae80c5ec0 con 0x7fbae8155420 2024-04-22T06:33:15.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.762+0000 7fbaddffb640 1 -- 172.21.15.44:0/285107267 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8004a20 con 0x7fbae8155420 2024-04-22T06:33:15.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.762+0000 7fbaef5bb640 1 -- 172.21.15.44:0/285107267 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbae8150270 con 0x7fbae8155420 2024-04-22T06:33:15.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.763+0000 7fbaef5bb640 1 -- 172.21.15.44:0/285107267 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbae81504b0 con 0x7fbae8155420 2024-04-22T06:33:15.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.763+0000 7fbaddffb640 1 -- 172.21.15.44:0/285107267 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbad801b070 con 0x7fbae8155420 2024-04-22T06:33:15.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.764+0000 7fbaddffb640 1 --2- 172.21.15.44:0/285107267 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fbacc075100 0x7fbacc0775c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:15.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.764+0000 7fbaddffb640 1 -- 172.21.15.44:0/285107267 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6821+0+0 (secure 0 0 0) 0x7fbad808b240 con 0x7fbae8155420 2024-04-22T06:33:15.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.764+0000 7fbaed357640 1 --2- 172.21.15.44:0/285107267 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fbacc075100 0x7fbacc0775c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:15.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.765+0000 7fbaed357640 1 --2- 172.21.15.44:0/285107267 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fbacc075100 0x7fbacc0775c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbae4009bd0 tx=0x7fbae4009240 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:15.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.768+0000 7fbaddffb640 1 -- 172.21.15.44:0/285107267 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbae81504b0 con 0x7fbae8155420 2024-04-22T06:33:15.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.955+0000 7fbaef5bb640 1 -- 172.21.15.44:0/285107267 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbae8069520 con 0x7fbae8155420 2024-04-22T06:33:15.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.956+0000 7fbaddffb640 1 -- 172.21.15.44:0/285107267 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18923 (secure 0 0 0) 0x7fbad8058b30 con 0x7fbae8155420 2024-04-22T06:33:15.955 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:33:15.955 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":28,"fsid":"9b0a34ca-403d-4513-8a66-d3749ba559aa","created":"2024-04-22T06:30:42.334581+0000","modified":"2024-04-22T06:33:07.139734+0000","last_up_change":"2024-04-22T06:30:52.942257+0000","last_in_change":"2024-04-22T06:30:44.467727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T06:30:54.560256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T06:31:04.612387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T06:31:05.312124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-22T06:31:09.025550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b4a6692-53c9-472d-b2f3-1dd29a6679ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6817","nonce":2220268493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6819","nonce":2220268493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6823","nonce":2220268493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2220268493},{"type":"v1","addr":"172.21.15.44:6821","nonce":2220268493}]},"public_addr":"172.21.15.44:6817/2220268493","cluster_addr":"172.21.15.44:6819/2220268493","heartbeat_back_addr":"172.21.15.44:6823/2220268493","heartbeat_front_addr":"172.21.15.44:6821/2220268493","state":["exists","up"]},{"osd":1,"uuid":"70a6370a-7c0a-4d64-a0e4-b2e726b4f291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6809","nonce":1278678977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6811","nonce":1278678977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6815","nonce":1278678977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":1278678977},{"type":"v1","addr":"172.21.15.44:6813","nonce":1278678977}]},"public_addr":"172.21.15.44:6809/1278678977","cluster_addr":"172.21.15.44:6811/1278678977","heartbeat_back_addr":"172.21.15.44:6815/1278678977","heartbeat_front_addr":"172.21.15.44:6813/1278678977","state":["exists","up"]},{"osd":2,"uuid":"f2e51a62-ca91-470d-bd5a-5710c61deee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6801","nonce":2544087903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6803","nonce":2544087903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6807","nonce":2544087903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2544087903},{"type":"v1","addr":"172.21.15.44:6805","nonce":2544087903}]},"public_addr":"172.21.15.44:6801/2544087903","cluster_addr":"172.21.15.44:6803/2544087903","heartbeat_back_addr":"172.21.15.44:6807/2544087903","heartbeat_front_addr":"172.21.15.44:6805/2544087903","state":["exists","up"]},{"osd":3,"uuid":"bc514203-9af9-4f1f-b18e-765bdd5de6e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6825","nonce":1692928109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6827","nonce":1692928109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6831","nonce":1692928109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1692928109},{"type":"v1","addr":"172.21.15.44:6829","nonce":1692928109}]},"public_addr":"172.21.15.44:6825/1692928109","cluster_addr":"172.21.15.44:6827/1692928109","heartbeat_back_addr":"172.21.15.44:6831/1692928109","heartbeat_front_addr":"172.21.15.44:6829/1692928109","state":["exists","up"]},{"osd":4,"uuid":"357c439a-6e08-4c19-ac18-b68e6c890497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6820","nonce":3802577427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6824","nonce":3802577427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6831","nonce":3802577427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":3802577427},{"type":"v1","addr":"172.21.15.70:6828","nonce":3802577427}]},"public_addr":"172.21.15.70:6820/3802577427","cluster_addr":"172.21.15.70:6824/3802577427","heartbeat_back_addr":"172.21.15.70:6831/3802577427","heartbeat_front_addr":"172.21.15.70:6828/3802577427","state":["exists","up"]},{"osd":5,"uuid":"952fb88f-d56e-4305-bfa8-d32674332dfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6801","nonce":664757345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6803","nonce":664757345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6807","nonce":664757345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":664757345},{"type":"v1","addr":"172.21.15.70:6805","nonce":664757345}]},"public_addr":"172.21.15.70:6801/664757345","cluster_addr":"172.21.15.70:6803/664757345","heartbeat_back_addr":"172.21.15.70:6807/664757345","heartbeat_front_addr":"172.21.15.70:6805/664757345","state":["exists","up"]},{"osd":6,"uuid":"74399b3d-fac2-45ab-9172-2cd95b02450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6817","nonce":1623253482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6821","nonce":1623253482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6827","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6829","nonce":1623253482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":1623253482},{"type":"v1","addr":"172.21.15.70:6825","nonce":1623253482}]},"public_addr":"172.21.15.70:6817/1623253482","cluster_addr":"172.21.15.70:6821/1623253482","heartbeat_back_addr":"172.21.15.70:6829/1623253482","heartbeat_front_addr":"172.21.15.70:6825/1623253482","state":["exists","up"]},{"osd":7,"uuid":"8f35508d-6fcc-43ff-8ee7-0498a7ab3b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6809","nonce":1551180300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6811","nonce":1551180300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6815","nonce":1551180300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":1551180300},{"type":"v1","addr":"172.21.15.70:6813","nonce":1551180300}]},"public_addr":"172.21.15.70:6809/1551180300","cluster_addr":"172.21.15.70:6811/1551180300","heartbeat_back_addr":"172.21.15.70:6815/1551180300","heartbeat_front_addr":"172.21.15.70:6813/1551180300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.612502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.653756+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.521169+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.649955+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.640231+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.686638+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.642199+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T06:30:49.700360+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6834/1497302315":"2024-04-23T06:33:07.139686+0000","172.21.15.44:6832/1497302315":"2024-04-23T06:33:07.139686+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T06:33:15.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.958+0000 7fbaef5bb640 1 -- 172.21.15.44:0/285107267 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fbacc075100 msgr2=0x7fbacc0775c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:15.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.958+0000 7fbaef5bb640 1 --2- 172.21.15.44:0/285107267 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fbacc075100 0x7fbacc0775c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbae4009bd0 tx=0x7fbae4009240 comp rx=0 tx=0).stop 2024-04-22T06:33:15.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.958+0000 7fbaef5bb640 1 -- 172.21.15.44:0/285107267 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbae8155420 msgr2=0x7fbae80c79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:15.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.958+0000 7fbaef5bb640 1 --2- 172.21.15.44:0/285107267 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbae8155420 0x7fbae80c79c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbad800c460 tx=0x7fbad800c920 comp rx=0 tx=0).stop 2024-04-22T06:33:15.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.959+0000 7fbaef5bb640 1 -- 172.21.15.44:0/285107267 shutdown_connections 2024-04-22T06:33:15.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.959+0000 7fbaef5bb640 1 --2- 172.21.15.44:0/285107267 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fbacc075100 0x7fbacc0775c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:15.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.959+0000 7fbaef5bb640 1 --2- 172.21.15.44:0/285107267 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbae815bea0 0x7fbae8153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:15.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.959+0000 7fbaef5bb640 1 --2- 172.21.15.44:0/285107267 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbae8155420 0x7fbae80c79c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:15.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.959+0000 7fbaef5bb640 1 --2- 172.21.15.44:0/285107267 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbae80c9c00 0x7fbae81f1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:15.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.959+0000 7fbaef5bb640 1 -- 172.21.15.44:0/285107267 >> 172.21.15.44:0/285107267 conn(0x7fbae8074cb0 msgr2=0x7fbae8159ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:33:15.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.959+0000 7fbaef5bb640 1 -- 172.21.15.44:0/285107267 shutdown_connections 2024-04-22T06:33:15.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:15.959+0000 7fbaef5bb640 1 -- 172.21.15.44:0/285107267 wait complete. 2024-04-22T06:33:15.968 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T06:30:54.560256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T06:31:04.612387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T06:31:05.312124+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-22T06:31:09.025550+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-22T06:33:15.969 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T06:33:16.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.081+0000 7f7052915640 1 Processor -- start 2024-04-22T06:33:16.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.081+0000 7f7052915640 1 -- start start 2024-04-22T06:33:16.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.082+0000 7f7052915640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f704c0c83f0 0x7f704c15baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:16.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.082+0000 7f7052915640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f704c074cb0 0x7f704c075160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:16.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.082+0000 7f7052915640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f704c143f30 con 0x7f704c1437f0 2024-04-22T06:33:16.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.082+0000 7f7052915640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f704c144760 con 0x7f704c0c83f0 2024-04-22T06:33:16.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.082+0000 7f7052915640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f704c0757b0 con 0x7f704c074cb0 2024-04-22T06:33:16.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.083+0000 7f704b7fe640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f704c0c83f0 0x7f704c15baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:16.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.083+0000 7f704bfff640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f704c1437f0 0x7f704c15f120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:35252/0 (socket says 172.21.15.44:35252) 2024-04-22T06:33:16.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.083+0000 7f704bfff640 1 -- 172.21.15.44:0/3454211460 learned_addr learned my addr 172.21.15.44:0/3454211460 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:33:16.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.083+0000 7f7050eb2640 1 --2- 172.21.15.44:0/3454211460 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f704c074cb0 0x7f704c075160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:16.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.083+0000 7f704b7fe640 1 -- 172.21.15.44:0/3454211460 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f704c074cb0 msgr2=0x7f704c075160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:33:16.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.083+0000 7f704b7fe640 1 --2- 172.21.15.44:0/3454211460 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f704c074cb0 0x7f704c075160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:16.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.083+0000 7f704b7fe640 1 -- 172.21.15.44:0/3454211460 >> v1:172.21.15.44:6789/0 conn(0x7f704c1437f0 legacy=0x7f704c15f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:16.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.083+0000 7f704affd640 1 -- 172.21.15.44:0/3454211460 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 594269295 0 0) 0x7f704c143f30 con 0x7f704c1437f0 2024-04-22T06:33:16.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.083+0000 7f704b7fe640 1 -- 172.21.15.44:0/3454211460 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f704c1441a0 con 0x7f704c0c83f0 2024-04-22T06:33:16.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.083+0000 7f7050eb2640 1 --2- 172.21.15.44:0/3454211460 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f704c074cb0 0x7f704c075160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:33:16.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.084+0000 7f704b7fe640 1 --2- 172.21.15.44:0/3454211460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f704c0c83f0 0x7f704c15baf0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f70400097a0 tx=0x7f704002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de90a66c5afc50c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:16.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.084+0000 7f704affd640 1 -- 172.21.15.44:0/3454211460 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7040034070 con 0x7f704c0c83f0 2024-04-22T06:33:16.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.084+0000 7f704affd640 1 -- 172.21.15.44:0/3454211460 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7040030610 con 0x7f704c0c83f0 2024-04-22T06:33:16.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.084+0000 7f704affd640 1 -- 172.21.15.44:0/3454211460 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70400309e0 con 0x7f704c0c83f0 2024-04-22T06:33:16.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.085+0000 7f7052915640 1 -- 172.21.15.44:0/3454211460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f704c0c83f0 msgr2=0x7f704c15baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:16.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.085+0000 7f7052915640 1 --2- 172.21.15.44:0/3454211460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f704c0c83f0 0x7f704c15baf0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f70400097a0 tx=0x7f704002b960 comp rx=0 tx=0).stop 2024-04-22T06:33:16.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.085+0000 7f7052915640 1 -- 172.21.15.44:0/3454211460 shutdown_connections 2024-04-22T06:33:16.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.085+0000 7f7052915640 1 --2- 172.21.15.44:0/3454211460 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f704c074cb0 0x7f704c075160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:16.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.085+0000 7f7052915640 1 --2- 172.21.15.44:0/3454211460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f704c0c83f0 0x7f704c15baf0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:16.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.085+0000 7f7052915640 1 -- 172.21.15.44:0/3454211460 >> 172.21.15.44:0/3454211460 conn(0x7f704c0cd630 msgr2=0x7f704c0cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:33:16.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.085+0000 7f7052915640 1 -- 172.21.15.44:0/3454211460 shutdown_connections 2024-04-22T06:33:16.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.085+0000 7f7052915640 1 -- 172.21.15.44:0/3454211460 wait complete. 2024-04-22T06:33:16.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.087+0000 7f7052915640 1 Processor -- start 2024-04-22T06:33:16.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.087+0000 7f7052915640 1 -- start start 2024-04-22T06:33:16.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.088+0000 7f7052915640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f704c074cb0 0x7f704c15e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:16.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.088+0000 7f7052915640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f704c0c83f0 0x7f704c13eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:16.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.088+0000 7f7052915640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f704c1437f0 0x7f704c13f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:16.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.088+0000 7f7052915640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f704c15b5d0 con 0x7f704c074cb0 2024-04-22T06:33:16.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.088+0000 7f7052915640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f704c15b2f0 con 0x7f704c0c83f0 2024-04-22T06:33:16.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.088+0000 7f7052915640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f704c15c030 con 0x7f704c1437f0 2024-04-22T06:33:16.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.088+0000 7f704bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f704c074cb0 0x7f704c15e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:16.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.088+0000 7f704bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f704c074cb0 0x7f704c15e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34066/0 (socket says 172.21.15.44:34066) 2024-04-22T06:33:16.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.088+0000 7f704bfff640 1 -- 172.21.15.44:0/253904626 learned_addr learned my addr 172.21.15.44:0/253904626 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:33:16.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.088+0000 7f704b7fe640 1 --2- 172.21.15.44:0/253904626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f704c0c83f0 0x7f704c13eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:16.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.088+0000 7f7050eb2640 1 --2- 172.21.15.44:0/253904626 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f704c1437f0 0x7f704c13f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:16.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.089+0000 7f704bfff640 1 -- 172.21.15.44:0/253904626 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f704c1437f0 msgr2=0x7f704c13f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:16.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.089+0000 7f704bfff640 1 --2- 172.21.15.44:0/253904626 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f704c1437f0 0x7f704c13f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:16.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.089+0000 7f704bfff640 1 -- 172.21.15.44:0/253904626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f704c0c83f0 msgr2=0x7f704c13eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:16.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.089+0000 7f704bfff640 1 --2- 172.21.15.44:0/253904626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f704c0c83f0 0x7f704c13eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:16.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.089+0000 7f704bfff640 1 -- 172.21.15.44:0/253904626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f704c1f1500 con 0x7f704c074cb0 2024-04-22T06:33:16.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.089+0000 7f704bfff640 1 --2- 172.21.15.44:0/253904626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f704c074cb0 0x7f704c15e7a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f703c00c460 tx=0x7f703c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:16.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.089+0000 7f7048ff9640 1 -- 172.21.15.44:0/253904626 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c018070 con 0x7f704c074cb0 2024-04-22T06:33:16.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.089+0000 7f7048ff9640 1 -- 172.21.15.44:0/253904626 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f703c007500 con 0x7f704c074cb0 2024-04-22T06:33:16.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.089+0000 7f7052915640 1 -- 172.21.15.44:0/253904626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f704c1e51b0 con 0x7f704c074cb0 2024-04-22T06:33:16.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.089+0000 7f7048ff9640 1 -- 172.21.15.44:0/253904626 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c004a20 con 0x7f704c074cb0 2024-04-22T06:33:16.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.089+0000 7f7052915640 1 -- 172.21.15.44:0/253904626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f704c1f17a0 con 0x7f704c074cb0 2024-04-22T06:33:16.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.090+0000 7f7052915640 1 -- 172.21.15.44:0/253904626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f704c069310 con 0x7f704c074cb0 2024-04-22T06:33:16.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.091+0000 7f7048ff9640 1 -- 172.21.15.44:0/253904626 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f703c01b070 con 0x7f704c074cb0 2024-04-22T06:33:16.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.092+0000 7f7048ff9640 1 --2- 172.21.15.44:0/253904626 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f703806c920 0x7f703806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:16.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.092+0000 7f7048ff9640 1 -- 172.21.15.44:0/253904626 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6821+0+0 (secure 0 0 0) 0x7f703c08b160 con 0x7f704c074cb0 2024-04-22T06:33:16.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.092+0000 7f704b7fe640 1 --2- 172.21.15.44:0/253904626 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f703806c920 0x7f703806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:16.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.092+0000 7f704b7fe640 1 --2- 172.21.15.44:0/253904626 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f703806c920 0x7f703806ede0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7040036000 tx=0x7f704003b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:16.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.096+0000 7f7048ff9640 1 -- 172.21.15.44:0/253904626 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f703c058ae0 con 0x7f704c074cb0 2024-04-22T06:33:16.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.285+0000 7f7052915640 1 -- 172.21.15.44:0/253904626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f704c0c81d0 con 0x7f704c074cb0 2024-04-22T06:33:16.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.286+0000 7f7048ff9640 1 -- 172.21.15.44:0/253904626 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7f703c05cde0 con 0x7f704c074cb0 2024-04-22T06:33:16.285 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-04-22T06:33:16.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.288+0000 7f7052915640 1 -- 172.21.15.44:0/253904626 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f703806c920 msgr2=0x7f703806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:16.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.288+0000 7f7052915640 1 --2- 172.21.15.44:0/253904626 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f703806c920 0x7f703806ede0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7040036000 tx=0x7f704003b040 comp rx=0 tx=0).stop 2024-04-22T06:33:16.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.288+0000 7f7052915640 1 -- 172.21.15.44:0/253904626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f704c074cb0 msgr2=0x7f704c15e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:16.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.288+0000 7f7052915640 1 --2- 172.21.15.44:0/253904626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f704c074cb0 0x7f704c15e7a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f703c00c460 tx=0x7f703c00c920 comp rx=0 tx=0).stop 2024-04-22T06:33:16.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.288+0000 7f7052915640 1 -- 172.21.15.44:0/253904626 shutdown_connections 2024-04-22T06:33:16.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.288+0000 7f7052915640 1 --2- 172.21.15.44:0/253904626 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f703806c920 0x7f703806ede0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:16.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.288+0000 7f7052915640 1 --2- 172.21.15.44:0/253904626 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f704c1437f0 0x7f704c13f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:16.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.288+0000 7f7052915640 1 --2- 172.21.15.44:0/253904626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f704c0c83f0 0x7f704c13eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:16.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.288+0000 7f7052915640 1 --2- 172.21.15.44:0/253904626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f704c074cb0 0x7f704c15e7a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:16.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.288+0000 7f7052915640 1 -- 172.21.15.44:0/253904626 >> 172.21.15.44:0/253904626 conn(0x7f704c0cd630 msgr2=0x7f704c0c9310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:33:16.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.288+0000 7f7052915640 1 -- 172.21.15.44:0/253904626 shutdown_connections 2024-04-22T06:33:16.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.288+0000 7f7052915640 1 -- 172.21.15.44:0/253904626 wait complete. 2024-04-22T06:33:16.299 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T06:33:16.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.406+0000 7fb60b4c0640 1 Processor -- start 2024-04-22T06:33:16.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.406+0000 7fb60b4c0640 1 -- start start 2024-04-22T06:33:16.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.407+0000 7fb60b4c0640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb604154f00 0x7fb6040cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:16.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.407+0000 7fb60b4c0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb6041552e0 0x7fb60415c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:16.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.407+0000 7fb60b4c0640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6040c5b30 con 0x7fb60406d470 2024-04-22T06:33:16.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.407+0000 7fb60b4c0640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fb6040c6740 con 0x7fb604154f00 2024-04-22T06:33:16.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.407+0000 7fb60b4c0640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fb6040c5e10 con 0x7fb6041552e0 2024-04-22T06:33:16.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.407+0000 7fb60925c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb604154f00 0x7fb6040cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:16.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.407+0000 7fb609a5d640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fb60406d470 0x7fb6040d0f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:35262/0 (socket says 172.21.15.44:35262) 2024-04-22T06:33:16.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.407+0000 7fb609a5d640 1 -- 172.21.15.44:0/2163335377 learned_addr learned my addr 172.21.15.44:0/2163335377 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:33:16.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.407+0000 7fb608a5b640 1 --2- 172.21.15.44:0/2163335377 >> v2:172.21.15.70:3300/0 conn(0x7fb6041552e0 0x7fb60415c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:16.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.408+0000 7fb60925c640 1 -- 172.21.15.44:0/2163335377 >> v2:172.21.15.70:3300/0 conn(0x7fb6041552e0 msgr2=0x7fb60415c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:33:16.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.408+0000 7fb60925c640 1 --2- 172.21.15.44:0/2163335377 >> v2:172.21.15.70:3300/0 conn(0x7fb6041552e0 0x7fb60415c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:16.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.408+0000 7fb60925c640 1 -- 172.21.15.44:0/2163335377 >> v1:172.21.15.44:6789/0 conn(0x7fb60406d470 legacy=0x7fb6040d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:16.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.408+0000 7fb60925c640 1 -- 172.21.15.44:0/2163335377 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6040c60f0 con 0x7fb604154f00 2024-04-22T06:33:16.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.408+0000 7fb60925c640 1 --2- 172.21.15.44:0/2163335377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb604154f00 0x7fb6040cb430 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb6040ce560 tx=0x7fb60002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cabf8e46a1390ac server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:16.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.409+0000 7fb5fbfff640 1 -- 172.21.15.44:0/2163335377 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6000315a0 con 0x7fb604154f00 2024-04-22T06:33:16.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.409+0000 7fb5fbfff640 1 -- 172.21.15.44:0/2163335377 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb600037070 con 0x7fb604154f00 2024-04-22T06:33:16.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.409+0000 7fb5fbfff640 1 -- 172.21.15.44:0/2163335377 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb600031960 con 0x7fb604154f00 2024-04-22T06:33:16.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.409+0000 7fb60b4c0640 1 -- 172.21.15.44:0/2163335377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb604154f00 msgr2=0x7fb6040cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:16.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.409+0000 7fb60b4c0640 1 --2- 172.21.15.44:0/2163335377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb604154f00 0x7fb6040cb430 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb6040ce560 tx=0x7fb60002b840 comp rx=0 tx=0).stop 2024-04-22T06:33:16.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.409+0000 7fb60b4c0640 1 -- 172.21.15.44:0/2163335377 shutdown_connections 2024-04-22T06:33:16.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.409+0000 7fb60b4c0640 1 --2- 172.21.15.44:0/2163335377 >> v2:172.21.15.70:3300/0 conn(0x7fb6041552e0 0x7fb60415c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:16.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.409+0000 7fb60b4c0640 1 --2- 172.21.15.44:0/2163335377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb604154f00 0x7fb6040cb430 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:16.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.409+0000 7fb60b4c0640 1 -- 172.21.15.44:0/2163335377 >> 172.21.15.44:0/2163335377 conn(0x7fb6040733c0 msgr2=0x7fb6040737d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:33:16.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.410+0000 7fb60b4c0640 1 -- 172.21.15.44:0/2163335377 shutdown_connections 2024-04-22T06:33:16.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.410+0000 7fb60b4c0640 1 -- 172.21.15.44:0/2163335377 wait complete. 2024-04-22T06:33:16.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.411+0000 7fb60b4c0640 1 Processor -- start 2024-04-22T06:33:16.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.411+0000 7fb60b4c0640 1 -- start start 2024-04-22T06:33:16.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.412+0000 7fb60b4c0640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb60406d470 0x7fb6040b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:16.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.412+0000 7fb60b4c0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb604154f00 0x7fb6040b3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:16.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.412+0000 7fb60b4c0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6041552e0 0x7fb60414c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:16.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.412+0000 7fb60b4c0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb6040c6d50 con 0x7fb6041552e0 2024-04-22T06:33:16.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.412+0000 7fb60b4c0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb6040c6bd0 con 0x7fb604154f00 2024-04-22T06:33:16.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.412+0000 7fb60b4c0640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb6040d1690 con 0x7fb60406d470 2024-04-22T06:33:16.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.412+0000 7fb609a5d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6041552e0 0x7fb60414c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:16.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.412+0000 7fb609a5d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6041552e0 0x7fb60414c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34086/0 (socket says 172.21.15.44:34086) 2024-04-22T06:33:16.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.412+0000 7fb609a5d640 1 -- 172.21.15.44:0/520421480 learned_addr learned my addr 172.21.15.44:0/520421480 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:33:16.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.412+0000 7fb60925c640 1 --2- 172.21.15.44:0/520421480 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb60406d470 0x7fb6040b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:16.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.412+0000 7fb608a5b640 1 --2- 172.21.15.44:0/520421480 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb604154f00 0x7fb6040b3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:16.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.412+0000 7fb609a5d640 1 -- 172.21.15.44:0/520421480 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb60406d470 msgr2=0x7fb6040b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:16.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.412+0000 7fb609a5d640 1 --2- 172.21.15.44:0/520421480 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb60406d470 0x7fb6040b5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:16.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.412+0000 7fb609a5d640 1 -- 172.21.15.44:0/520421480 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb604154f00 msgr2=0x7fb6040b3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:16.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.412+0000 7fb609a5d640 1 --2- 172.21.15.44:0/520421480 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb604154f00 0x7fb6040b3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:16.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.412+0000 7fb609a5d640 1 -- 172.21.15.44:0/520421480 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6041f0d30 con 0x7fb6041552e0 2024-04-22T06:33:16.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.412+0000 7fb608a5b640 1 --2- 172.21.15.44:0/520421480 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb604154f00 0x7fb6040b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T06:33:16.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.413+0000 7fb609a5d640 1 --2- 172.21.15.44:0/520421480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6041552e0 0x7fb60414c2c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc00ca80 tx=0x7fb5fc00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:16.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.413+0000 7fb5f9ffb640 1 -- 172.21.15.44:0/520421480 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc017070 con 0x7fb6041552e0 2024-04-22T06:33:16.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.413+0000 7fb5f9ffb640 1 -- 172.21.15.44:0/520421480 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5fc00ac30 con 0x7fb6041552e0 2024-04-22T06:33:16.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.413+0000 7fb5f9ffb640 1 -- 172.21.15.44:0/520421480 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc012600 con 0x7fb6041552e0 2024-04-22T06:33:16.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.413+0000 7fb60b4c0640 1 -- 172.21.15.44:0/520421480 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6041f4d50 con 0x7fb6041552e0 2024-04-22T06:33:16.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.413+0000 7fb60b4c0640 1 -- 172.21.15.44:0/520421480 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6041f12d0 con 0x7fb6041552e0 2024-04-22T06:33:16.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.414+0000 7fb60b4c0640 1 -- 172.21.15.44:0/520421480 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb604068cc0 con 0x7fb6041552e0 2024-04-22T06:33:16.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.415+0000 7fb5f9ffb640 1 -- 172.21.15.44:0/520421480 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb5fc01a070 con 0x7fb6041552e0 2024-04-22T06:33:16.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.416+0000 7fb5f9ffb640 1 --2- 172.21.15.44:0/520421480 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb5e806c8f0 0x7fb5e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:16.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.416+0000 7fb5f9ffb640 1 -- 172.21.15.44:0/520421480 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6821+0+0 (secure 0 0 0) 0x7fb5fc08bb60 con 0x7fb6041552e0 2024-04-22T06:33:16.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.416+0000 7fb60925c640 1 --2- 172.21.15.44:0/520421480 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb5e806c8f0 0x7fb5e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:16.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.417+0000 7fb60925c640 1 --2- 172.21.15.44:0/520421480 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb5e806c8f0 0x7fb5e806edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb600036000 tx=0x7fb60003d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:16.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.424+0000 7fb5f9ffb640 1 -- 172.21.15.44:0/520421480 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb5fc00ae20 con 0x7fb6041552e0 2024-04-22T06:33:16.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.608+0000 7fb60b4c0640 1 -- 172.21.15.44:0/520421480 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb6040caee0 con 0x7fb6041552e0 2024-04-22T06:33:16.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.608+0000 7fb5f9ffb640 1 -- 172.21.15.44:0/520421480 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7fb5fc0594e0 con 0x7fb6041552e0 2024-04-22T06:33:16.607 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-04-22T06:33:16.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.610+0000 7fb60b4c0640 1 -- 172.21.15.44:0/520421480 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb5e806c8f0 msgr2=0x7fb5e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:16.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.610+0000 7fb60b4c0640 1 --2- 172.21.15.44:0/520421480 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb5e806c8f0 0x7fb5e806edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb600036000 tx=0x7fb60003d040 comp rx=0 tx=0).stop 2024-04-22T06:33:16.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.610+0000 7fb60b4c0640 1 -- 172.21.15.44:0/520421480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6041552e0 msgr2=0x7fb60414c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:16.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.610+0000 7fb60b4c0640 1 --2- 172.21.15.44:0/520421480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6041552e0 0x7fb60414c2c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc00ca80 tx=0x7fb5fc00cf40 comp rx=0 tx=0).stop 2024-04-22T06:33:16.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.610+0000 7fb60b4c0640 1 -- 172.21.15.44:0/520421480 shutdown_connections 2024-04-22T06:33:16.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.610+0000 7fb60b4c0640 1 --2- 172.21.15.44:0/520421480 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7fb5e806c8f0 0x7fb5e806edb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:16.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.610+0000 7fb60b4c0640 1 --2- 172.21.15.44:0/520421480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6041552e0 0x7fb60414c2c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:16.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.610+0000 7fb60b4c0640 1 --2- 172.21.15.44:0/520421480 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb604154f00 0x7fb6040b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:16.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.610+0000 7fb60b4c0640 1 --2- 172.21.15.44:0/520421480 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb60406d470 0x7fb6040b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:16.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.610+0000 7fb60b4c0640 1 -- 172.21.15.44:0/520421480 >> 172.21.15.44:0/520421480 conn(0x7fb6040733c0 msgr2=0x7fb60414fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:33:16.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.611+0000 7fb60b4c0640 1 -- 172.21.15.44:0/520421480 shutdown_connections 2024-04-22T06:33:16.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.611+0000 7fb60b4c0640 1 -- 172.21.15.44:0/520421480 wait complete. 2024-04-22T06:33:16.621 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T06:33:16.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.728+0000 7f69e19f4640 1 Processor -- start 2024-04-22T06:33:16.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.728+0000 7f69e19f4640 1 -- start start 2024-04-22T06:33:16.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.729+0000 7f69e19f4640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f69dc06bf80 0x7f69dc073b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:16.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.729+0000 7f69e19f4640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f69dc0cf360 0x7f69dc0ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:16.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.729+0000 7f69e19f4640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69dc06d500 con 0x7f69dc06bba0 2024-04-22T06:33:16.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.729+0000 7f69e19f4640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f69dc0c9d50 con 0x7f69dc0cf360 2024-04-22T06:33:16.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.729+0000 7f69e19f4640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f69dc070b00 con 0x7f69dc06bf80 2024-04-22T06:33:16.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.730+0000 7f69daffd640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f69dc06bba0 0x7f69dc155150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:35268/0 (socket says 172.21.15.44:35268) 2024-04-22T06:33:16.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.730+0000 7f69daffd640 1 -- 172.21.15.44:0/2343523397 learned_addr learned my addr 172.21.15.44:0/2343523397 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:33:16.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.730+0000 7f69db7fe640 1 --2- 172.21.15.44:0/2343523397 >> v2:172.21.15.44:3300/0 conn(0x7f69dc0cf360 0x7f69dc0ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:16.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.730+0000 7f69da7fc640 1 --2- 172.21.15.44:0/2343523397 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f69dc06bf80 0x7f69dc073b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:16.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.730+0000 7f69d9ffb640 1 -- 172.21.15.44:0/2343523397 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 871243081 0 0) 0x7f69dc06d500 con 0x7f69dc06bba0 2024-04-22T06:33:16.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.730+0000 7f69d9ffb640 1 -- 172.21.15.44:0/2343523397 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f69c4003520 con 0x7f69dc06bba0 2024-04-22T06:33:16.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.730+0000 7f69db7fe640 1 -- 172.21.15.44:0/2343523397 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f69dc06bf80 msgr2=0x7f69dc073b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:33:16.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.730+0000 7f69db7fe640 1 --2- 172.21.15.44:0/2343523397 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f69dc06bf80 0x7f69dc073b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:16.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.730+0000 7f69db7fe640 1 -- 172.21.15.44:0/2343523397 >> v1:172.21.15.44:6789/0 conn(0x7f69dc06bba0 legacy=0x7f69dc155150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:16.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.730+0000 7f69db7fe640 1 -- 172.21.15.44:0/2343523397 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69dc070de0 con 0x7f69dc0cf360 2024-04-22T06:33:16.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.731+0000 7f69da7fc640 1 --2- 172.21.15.44:0/2343523397 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f69dc06bf80 0x7f69dc073b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:33:16.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.731+0000 7f69db7fe640 1 --2- 172.21.15.44:0/2343523397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69dc0cf360 0x7f69dc0ca1e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f69d4009dc0 tx=0x7f69d402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=186250c6dbc18930 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:16.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.731+0000 7f69d9ffb640 1 -- 172.21.15.44:0/2343523397 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d4039070 con 0x7f69dc0cf360 2024-04-22T06:33:16.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.731+0000 7f69d9ffb640 1 -- 172.21.15.44:0/2343523397 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69d402cc60 con 0x7f69dc0cf360 2024-04-22T06:33:16.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.731+0000 7f69d9ffb640 1 -- 172.21.15.44:0/2343523397 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d4034640 con 0x7f69dc0cf360 2024-04-22T06:33:16.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.732+0000 7f69e19f4640 1 -- 172.21.15.44:0/2343523397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69dc0cf360 msgr2=0x7f69dc0ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:16.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.732+0000 7f69e19f4640 1 --2- 172.21.15.44:0/2343523397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69dc0cf360 0x7f69dc0ca1e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f69d4009dc0 tx=0x7f69d402beb0 comp rx=0 tx=0).stop 2024-04-22T06:33:16.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.732+0000 7f69e19f4640 1 -- 172.21.15.44:0/2343523397 shutdown_connections 2024-04-22T06:33:16.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.732+0000 7f69e19f4640 1 --2- 172.21.15.44:0/2343523397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69dc0cf360 0x7f69dc0ca1e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:16.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.732+0000 7f69e19f4640 1 --2- 172.21.15.44:0/2343523397 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f69dc06bf80 0x7f69dc073b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:16.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.732+0000 7f69e19f4640 1 -- 172.21.15.44:0/2343523397 >> 172.21.15.44:0/2343523397 conn(0x7f69dc0ce7c0 msgr2=0x7f69dc0ceba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:33:16.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.732+0000 7f69e19f4640 1 -- 172.21.15.44:0/2343523397 shutdown_connections 2024-04-22T06:33:16.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.732+0000 7f69e19f4640 1 -- 172.21.15.44:0/2343523397 wait complete. 2024-04-22T06:33:16.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.734+0000 7f69e19f4640 1 Processor -- start 2024-04-22T06:33:16.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.734+0000 7f69e19f4640 1 -- start start 2024-04-22T06:33:16.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.734+0000 7f69e19f4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69dc06bba0 0x7f69dc1f0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:16.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.734+0000 7f69e19f4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69dc06bf80 0x7f69dc15aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:16.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.734+0000 7f69e19f4640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f69dc0cf360 0x7f69dc1e7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:16.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.735+0000 7f69e19f4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f69dc0717f0 con 0x7f69dc06bba0 2024-04-22T06:33:16.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.735+0000 7f69e19f4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f69dc0713a0 con 0x7f69dc06bf80 2024-04-22T06:33:16.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.735+0000 7f69e19f4640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f69dc0c9140 con 0x7f69dc0cf360 2024-04-22T06:33:16.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.735+0000 7f69daffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69dc06bba0 0x7f69dc1f0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:16.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.735+0000 7f69daffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69dc06bba0 0x7f69dc1f0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34098/0 (socket says 172.21.15.44:34098) 2024-04-22T06:33:16.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.735+0000 7f69daffd640 1 -- 172.21.15.44:0/2421725755 learned_addr learned my addr 172.21.15.44:0/2421725755 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:33:16.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.735+0000 7f69db7fe640 1 --2- 172.21.15.44:0/2421725755 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f69dc0cf360 0x7f69dc1e7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:16.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.735+0000 7f69da7fc640 1 --2- 172.21.15.44:0/2421725755 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69dc06bf80 0x7f69dc15aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:16.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.735+0000 7f69daffd640 1 -- 172.21.15.44:0/2421725755 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f69dc0cf360 msgr2=0x7f69dc1e7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:16.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.735+0000 7f69daffd640 1 --2- 172.21.15.44:0/2421725755 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f69dc0cf360 0x7f69dc1e7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:16.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.735+0000 7f69daffd640 1 -- 172.21.15.44:0/2421725755 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69dc06bf80 msgr2=0x7f69dc15aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:16.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.735+0000 7f69daffd640 1 --2- 172.21.15.44:0/2421725755 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69dc06bf80 0x7f69dc15aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:16.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.735+0000 7f69daffd640 1 -- 172.21.15.44:0/2421725755 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69dc159750 con 0x7f69dc06bba0 2024-04-22T06:33:16.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.735+0000 7f69daffd640 1 --2- 172.21.15.44:0/2421725755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69dc06bba0 0x7f69dc1f0e60 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f69cc00c500 tx=0x7f69cc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:16.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.735+0000 7f69c3fff640 1 -- 172.21.15.44:0/2421725755 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc017070 con 0x7f69dc06bba0 2024-04-22T06:33:16.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.735+0000 7f69c3fff640 1 -- 172.21.15.44:0/2421725755 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69cc005d00 con 0x7f69dc06bba0 2024-04-22T06:33:16.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.735+0000 7f69c3fff640 1 -- 172.21.15.44:0/2421725755 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc007330 con 0x7f69dc06bba0 2024-04-22T06:33:16.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.736+0000 7f69e19f4640 1 -- 172.21.15.44:0/2421725755 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69dc159a30 con 0x7f69dc06bba0 2024-04-22T06:33:16.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.736+0000 7f69e19f4640 1 -- 172.21.15.44:0/2421725755 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69dc159ff0 con 0x7f69dc06bba0 2024-04-22T06:33:16.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.737+0000 7f69e19f4640 1 -- 172.21.15.44:0/2421725755 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69dc069310 con 0x7f69dc06bba0 2024-04-22T06:33:16.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.738+0000 7f69c3fff640 1 -- 172.21.15.44:0/2421725755 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f69cc01a070 con 0x7f69dc06bba0 2024-04-22T06:33:16.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.738+0000 7f69c3fff640 1 --2- 172.21.15.44:0/2421725755 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f69c806c8f0 0x7f69c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:16.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.738+0000 7f69c3fff640 1 -- 172.21.15.44:0/2421725755 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6821+0+0 (secure 0 0 0) 0x7f69cc08b2c0 con 0x7f69dc06bba0 2024-04-22T06:33:16.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.739+0000 7f69da7fc640 1 --2- 172.21.15.44:0/2421725755 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f69c806c8f0 0x7f69c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:16.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.739+0000 7f69da7fc640 1 --2- 172.21.15.44:0/2421725755 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f69c806c8f0 0x7f69c806edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f69dc0ca6a0 tx=0x7f69d0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:16.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.743+0000 7f69c3fff640 1 -- 172.21.15.44:0/2421725755 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f69cc058cd0 con 0x7f69dc06bba0 2024-04-22T06:33:16.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.927+0000 7f69e19f4640 1 -- 172.21.15.44:0/2421725755 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f69dc071a60 con 0x7f69dc06bba0 2024-04-22T06:33:16.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.927+0000 7f69c3fff640 1 -- 172.21.15.44:0/2421725755 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7f69cc05cfd0 con 0x7f69dc06bba0 2024-04-22T06:33:16.926 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-04-22T06:33:16.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.929+0000 7f69e19f4640 1 -- 172.21.15.44:0/2421725755 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f69c806c8f0 msgr2=0x7f69c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:16.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.929+0000 7f69e19f4640 1 --2- 172.21.15.44:0/2421725755 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f69c806c8f0 0x7f69c806edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f69dc0ca6a0 tx=0x7f69d0008040 comp rx=0 tx=0).stop 2024-04-22T06:33:16.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.929+0000 7f69e19f4640 1 -- 172.21.15.44:0/2421725755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69dc06bba0 msgr2=0x7f69dc1f0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:16.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.929+0000 7f69e19f4640 1 --2- 172.21.15.44:0/2421725755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69dc06bba0 0x7f69dc1f0e60 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f69cc00c500 tx=0x7f69cc00c9c0 comp rx=0 tx=0).stop 2024-04-22T06:33:16.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.929+0000 7f69e19f4640 1 -- 172.21.15.44:0/2421725755 shutdown_connections 2024-04-22T06:33:16.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.929+0000 7f69e19f4640 1 --2- 172.21.15.44:0/2421725755 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f69c806c8f0 0x7f69c806edb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:16.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.929+0000 7f69e19f4640 1 --2- 172.21.15.44:0/2421725755 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f69dc0cf360 0x7f69dc1e7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:16.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.929+0000 7f69e19f4640 1 --2- 172.21.15.44:0/2421725755 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69dc06bf80 0x7f69dc15aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:16.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.929+0000 7f69e19f4640 1 --2- 172.21.15.44:0/2421725755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69dc06bba0 0x7f69dc1f0e60 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:16.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.929+0000 7f69e19f4640 1 -- 172.21.15.44:0/2421725755 >> 172.21.15.44:0/2421725755 conn(0x7f69dc0ce7c0 msgr2=0x7f69dc154f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:33:16.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.929+0000 7f69e19f4640 1 -- 172.21.15.44:0/2421725755 shutdown_connections 2024-04-22T06:33:16.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:16.929+0000 7f69e19f4640 1 -- 172.21.15.44:0/2421725755 wait complete. 2024-04-22T06:33:16.939 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-22T06:33:17.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.051+0000 7f6f71df4640 1 Processor -- start 2024-04-22T06:33:17.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.051+0000 7f6f71df4640 1 -- start start 2024-04-22T06:33:17.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.052+0000 7f6f71df4640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6f6c0c4300 0x7f6f6c158970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:17.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.052+0000 7f6f71df4640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f6f6c073780 0x7f6f6c15b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:17.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.052+0000 7f6f71df4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f6c072160 con 0x7f6f6c14e810 2024-04-22T06:33:17.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.052+0000 7f6f71df4640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f6f6c1584e0 con 0x7f6f6c073780 2024-04-22T06:33:17.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.052+0000 7f6f71df4640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f6f6c072400 con 0x7f6f6c0c4300 2024-04-22T06:33:17.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.052+0000 7f6f6bfff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f6f6c073780 0x7f6f6c15b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:17.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.052+0000 7f6f6bfff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f6f6c073780 0x7f6f6c15b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34110/0 (socket says 172.21.15.44:34110) 2024-04-22T06:33:17.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.052+0000 7f6f6bfff640 1 -- 172.21.15.44:0/2158313453 learned_addr learned my addr 172.21.15.44:0/2158313453 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:33:17.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.052+0000 7f6f6affd640 1 --2- 172.21.15.44:0/2158313453 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6f6c0c4300 0x7f6f6c158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:17.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.052+0000 7f6f6bfff640 1 -- 172.21.15.44:0/2158313453 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6f6c0c4300 msgr2=0x7f6f6c158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:33:17.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.052+0000 7f6f6bfff640 1 --2- 172.21.15.44:0/2158313453 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6f6c0c4300 0x7f6f6c158970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:17.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.052+0000 7f6f6bfff640 1 -- 172.21.15.44:0/2158313453 >> v1:172.21.15.70:6789/0 conn(0x7f6f6c14e810 legacy=0x7f6f6c072c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:17.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.052+0000 7f6f6a7fc640 1 -- 172.21.15.44:0/2158313453 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1785604312 0 0) 0x7f6f6c072160 con 0x7f6f6c14e810 2024-04-22T06:33:17.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.052+0000 7f6f6bfff640 1 -- 172.21.15.44:0/2158313453 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f6c0726e0 con 0x7f6f6c073780 2024-04-22T06:33:17.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.053+0000 7f6f6bfff640 1 --2- 172.21.15.44:0/2158313453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f6c073780 0x7f6f6c15b410 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6f64009e50 tx=0x7f6f6402bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45617db4f183c67 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:17.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.053+0000 7f6f6a7fc640 1 -- 172.21.15.44:0/2158313453 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6402cb90 con 0x7f6f6c073780 2024-04-22T06:33:17.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.053+0000 7f6f6a7fc640 1 -- 172.21.15.44:0/2158313453 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f6402cd30 con 0x7f6f6c073780 2024-04-22T06:33:17.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.053+0000 7f6f6a7fc640 1 -- 172.21.15.44:0/2158313453 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f64034610 con 0x7f6f6c073780 2024-04-22T06:33:17.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.054+0000 7f6f71df4640 1 -- 172.21.15.44:0/2158313453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f6c073780 msgr2=0x7f6f6c15b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:17.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.054+0000 7f6f71df4640 1 --2- 172.21.15.44:0/2158313453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f6c073780 0x7f6f6c15b410 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6f64009e50 tx=0x7f6f6402bff0 comp rx=0 tx=0).stop 2024-04-22T06:33:17.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.054+0000 7f6f71df4640 1 -- 172.21.15.44:0/2158313453 shutdown_connections 2024-04-22T06:33:17.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.054+0000 7f6f71df4640 1 --2- 172.21.15.44:0/2158313453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f6c073780 0x7f6f6c15b410 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:17.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.054+0000 7f6f71df4640 1 --2- 172.21.15.44:0/2158313453 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6f6c0c4300 0x7f6f6c158970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:17.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.054+0000 7f6f71df4640 1 -- 172.21.15.44:0/2158313453 >> 172.21.15.44:0/2158313453 conn(0x7f6f6c06e330 msgr2=0x7f6f6c06b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:33:17.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.054+0000 7f6f71df4640 1 -- 172.21.15.44:0/2158313453 shutdown_connections 2024-04-22T06:33:17.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.054+0000 7f6f71df4640 1 -- 172.21.15.44:0/2158313453 wait complete. 2024-04-22T06:33:17.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.055+0000 7f6f71df4640 1 Processor -- start 2024-04-22T06:33:17.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.056+0000 7f6f71df4640 1 -- start start 2024-04-22T06:33:17.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.056+0000 7f6f71df4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f6c073780 0x7f6f6c0c7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:17.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.056+0000 7f6f71df4640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6f6c0c4300 0x7f6f6c13df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:17.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.056+0000 7f6f71df4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f6c14e810 0x7f6f6c0c5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:17.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.056+0000 7f6f71df4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c14ebf0 con 0x7f6f6c073780 2024-04-22T06:33:17.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.056+0000 7f6f71df4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c157e20 con 0x7f6f6c14e810 2024-04-22T06:33:17.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.056+0000 7f6f71df4640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f6f6c073350 con 0x7f6f6c0c4300 2024-04-22T06:33:17.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.056+0000 7f6f6b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f6c073780 0x7f6f6c0c7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:17.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.056+0000 7f6f6b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f6c073780 0x7f6f6c0c7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34118/0 (socket says 172.21.15.44:34118) 2024-04-22T06:33:17.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.056+0000 7f6f6b7fe640 1 -- 172.21.15.44:0/1686040390 learned_addr learned my addr 172.21.15.44:0/1686040390 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:33:17.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.056+0000 7f6f6affd640 1 --2- 172.21.15.44:0/1686040390 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6f6c0c4300 0x7f6f6c13df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:17.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.057+0000 7f6f6bfff640 1 --2- 172.21.15.44:0/1686040390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f6c14e810 0x7f6f6c0c5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:17.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.057+0000 7f6f6b7fe640 1 -- 172.21.15.44:0/1686040390 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6f6c0c4300 msgr2=0x7f6f6c13df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:17.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.057+0000 7f6f6b7fe640 1 --2- 172.21.15.44:0/1686040390 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6f6c0c4300 0x7f6f6c13df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:17.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.057+0000 7f6f6b7fe640 1 -- 172.21.15.44:0/1686040390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f6c14e810 msgr2=0x7f6f6c0c5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:17.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.057+0000 7f6f6b7fe640 1 --2- 172.21.15.44:0/1686040390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f6c14e810 0x7f6f6c0c5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:17.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.057+0000 7f6f6b7fe640 1 -- 172.21.15.44:0/1686040390 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f6c13ec60 con 0x7f6f6c073780 2024-04-22T06:33:17.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.057+0000 7f6f6affd640 1 --2- 172.21.15.44:0/1686040390 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6f6c0c4300 0x7f6f6c13df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:33:17.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.057+0000 7f6f6b7fe640 1 --2- 172.21.15.44:0/1686040390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f6c073780 0x7f6f6c0c7400 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c00c440 tx=0x7f6f5c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:17.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.057+0000 7f6f53fff640 1 -- 172.21.15.44:0/1686040390 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c017070 con 0x7f6f6c073780 2024-04-22T06:33:17.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.057+0000 7f6f71df4640 1 -- 172.21.15.44:0/1686040390 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f6c159080 con 0x7f6f6c073780 2024-04-22T06:33:17.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.057+0000 7f6f53fff640 1 -- 172.21.15.44:0/1686040390 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f5c005480 con 0x7f6f6c073780 2024-04-22T06:33:17.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.057+0000 7f6f53fff640 1 -- 172.21.15.44:0/1686040390 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c017070 con 0x7f6f6c073780 2024-04-22T06:33:17.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.057+0000 7f6f71df4640 1 -- 172.21.15.44:0/1686040390 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f6c15af10 con 0x7f6f6c073780 2024-04-22T06:33:17.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.059+0000 7f6f71df4640 1 -- 172.21.15.44:0/1686040390 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f6c068cc0 con 0x7f6f6c073780 2024-04-22T06:33:17.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.060+0000 7f6f53fff640 1 -- 172.21.15.44:0/1686040390 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6f5c01a070 con 0x7f6f6c073780 2024-04-22T06:33:17.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.060+0000 7f6f53fff640 1 --2- 172.21.15.44:0/1686040390 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f6f5806caf0 0x7f6f5806efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:17.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.060+0000 7f6f53fff640 1 -- 172.21.15.44:0/1686040390 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6821+0+0 (secure 0 0 0) 0x7f6f5c08aed0 con 0x7f6f6c073780 2024-04-22T06:33:17.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.060+0000 7f6f6affd640 1 --2- 172.21.15.44:0/1686040390 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f6f5806caf0 0x7f6f5806efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:17.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.061+0000 7f6f6affd640 1 --2- 172.21.15.44:0/1686040390 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f6f5806caf0 0x7f6f5806efb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c15b730 tx=0x7f6f60009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:17.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.068+0000 7f6f53fff640 1 -- 172.21.15.44:0/1686040390 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6f5c058740 con 0x7f6f6c073780 2024-04-22T06:33:17.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.257+0000 7f6f71df4640 1 -- 172.21.15.44:0/1686040390 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f6f6c15b110 con 0x7f6f6c073780 2024-04-22T06:33:17.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.257+0000 7f6f53fff640 1 -- 172.21.15.44:0/1686040390 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7f6f5c05ca40 con 0x7f6f6c073780 2024-04-22T06:33:17.257 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-04-22T06:33:17.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.259+0000 7f6f71df4640 1 -- 172.21.15.44:0/1686040390 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f6f5806caf0 msgr2=0x7f6f5806efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:17.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.259+0000 7f6f71df4640 1 --2- 172.21.15.44:0/1686040390 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f6f5806caf0 0x7f6f5806efb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c15b730 tx=0x7f6f60009190 comp rx=0 tx=0).stop 2024-04-22T06:33:17.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.259+0000 7f6f71df4640 1 -- 172.21.15.44:0/1686040390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f6c073780 msgr2=0x7f6f6c0c7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:17.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.259+0000 7f6f71df4640 1 --2- 172.21.15.44:0/1686040390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f6c073780 0x7f6f6c0c7400 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c00c440 tx=0x7f6f5c00c900 comp rx=0 tx=0).stop 2024-04-22T06:33:17.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.260+0000 7f6f71df4640 1 -- 172.21.15.44:0/1686040390 shutdown_connections 2024-04-22T06:33:17.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.260+0000 7f6f71df4640 1 --2- 172.21.15.44:0/1686040390 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f6f5806caf0 0x7f6f5806efb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:17.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.260+0000 7f6f71df4640 1 --2- 172.21.15.44:0/1686040390 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6f6c0c4300 0x7f6f6c13df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:17.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.260+0000 7f6f71df4640 1 --2- 172.21.15.44:0/1686040390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f6c14e810 0x7f6f6c0c5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:17.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.260+0000 7f6f71df4640 1 --2- 172.21.15.44:0/1686040390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f6c073780 0x7f6f6c0c7400 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:17.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.260+0000 7f6f71df4640 1 -- 172.21.15.44:0/1686040390 >> 172.21.15.44:0/1686040390 conn(0x7f6f6c06e330 msgr2=0x7f6f6c149990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:33:17.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.260+0000 7f6f71df4640 1 -- 172.21.15.44:0/1686040390 shutdown_connections 2024-04-22T06:33:17.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.260+0000 7f6f71df4640 1 -- 172.21.15.44:0/1686040390 wait complete. 2024-04-22T06:33:17.273 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-22T06:33:17.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.382+0000 7f155d2ba640 1 Processor -- start 2024-04-22T06:33:17.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.383+0000 7f155d2ba640 1 -- start start 2024-04-22T06:33:17.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.383+0000 7f155d2ba640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f15581552e0 0x7f15580cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:17.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.383+0000 7f155d2ba640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f155806d470 0x7f15580d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:17.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.383+0000 7f155d2ba640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f155815ccf0 con 0x7f1558154f00 2024-04-22T06:33:17.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.383+0000 7f155d2ba640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f15580c6740 con 0x7f155806d470 2024-04-22T06:33:17.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.383+0000 7f155d2ba640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f15580c5b30 con 0x7f15581552e0 2024-04-22T06:33:17.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.384+0000 7f1556ffd640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f1558154f00 0x7f155815c5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:35280/0 (socket says 172.21.15.44:35280) 2024-04-22T06:33:17.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.384+0000 7f1556ffd640 1 -- 172.21.15.44:0/203487183 learned_addr learned my addr 172.21.15.44:0/203487183 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T06:33:17.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.384+0000 7f15577fe640 1 --2- 172.21.15.44:0/203487183 >> v2:172.21.15.44:3300/0 conn(0x7f155806d470 0x7f15580d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:17.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.384+0000 7f15567fc640 1 --2- 172.21.15.44:0/203487183 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f15581552e0 0x7f15580cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:17.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.384+0000 7f15577fe640 1 -- 172.21.15.44:0/203487183 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f15581552e0 msgr2=0x7f15580cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T06:33:17.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.384+0000 7f15577fe640 1 --2- 172.21.15.44:0/203487183 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f15581552e0 0x7f15580cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:17.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.384+0000 7f15577fe640 1 -- 172.21.15.44:0/203487183 >> v1:172.21.15.44:6789/0 conn(0x7f1558154f00 legacy=0x7f155815c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:17.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.384+0000 7f15577fe640 1 -- 172.21.15.44:0/203487183 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15580c5e10 con 0x7f155806d470 2024-04-22T06:33:17.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.385+0000 7f15577fe640 1 --2- 172.21.15.44:0/203487183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f155806d470 0x7f15580d0f80 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f1550009d90 tx=0x7f155002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e22f8b76b61fade6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:17.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.385+0000 7f1555ffb640 1 -- 172.21.15.44:0/203487183 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1550039070 con 0x7f155806d470 2024-04-22T06:33:17.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.385+0000 7f1555ffb640 1 -- 172.21.15.44:0/203487183 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f155002cc60 con 0x7f155806d470 2024-04-22T06:33:17.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.385+0000 7f1555ffb640 1 -- 172.21.15.44:0/203487183 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1550034660 con 0x7f155806d470 2024-04-22T06:33:17.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.386+0000 7f155d2ba640 1 -- 172.21.15.44:0/203487183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f155806d470 msgr2=0x7f15580d0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:17.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.386+0000 7f155d2ba640 1 --2- 172.21.15.44:0/203487183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f155806d470 0x7f15580d0f80 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f1550009d90 tx=0x7f155002beb0 comp rx=0 tx=0).stop 2024-04-22T06:33:17.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.386+0000 7f155d2ba640 1 -- 172.21.15.44:0/203487183 shutdown_connections 2024-04-22T06:33:17.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.386+0000 7f155d2ba640 1 --2- 172.21.15.44:0/203487183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f155806d470 0x7f15580d0f80 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:17.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.386+0000 7f155d2ba640 1 --2- 172.21.15.44:0/203487183 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f15581552e0 0x7f15580cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:17.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.386+0000 7f155d2ba640 1 -- 172.21.15.44:0/203487183 >> 172.21.15.44:0/203487183 conn(0x7f15580733c0 msgr2=0x7f15580737d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:33:17.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.386+0000 7f155d2ba640 1 -- 172.21.15.44:0/203487183 shutdown_connections 2024-04-22T06:33:17.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.386+0000 7f155d2ba640 1 -- 172.21.15.44:0/203487183 wait complete. 2024-04-22T06:33:17.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.387+0000 7f155d2ba640 1 Processor -- start 2024-04-22T06:33:17.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.387+0000 7f155d2ba640 1 -- start start 2024-04-22T06:33:17.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.388+0000 7f155d2ba640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f155806d470 0x7f1558159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:17.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.388+0000 7f155d2ba640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1558154f00 0x7f15580b54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:17.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.388+0000 7f155d2ba640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f15581552e0 0x7f155815e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:17.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.388+0000 7f155d2ba640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f15580c6bd0 con 0x7f155806d470 2024-04-22T06:33:17.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.388+0000 7f155d2ba640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f15580c63d0 con 0x7f1558154f00 2024-04-22T06:33:17.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.388+0000 7f155d2ba640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f15580c6d50 con 0x7f15581552e0 2024-04-22T06:33:17.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.388+0000 7f1556ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f155806d470 0x7f1558159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:17.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.388+0000 7f1556ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f155806d470 0x7f1558159d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34144/0 (socket says 172.21.15.44:34144) 2024-04-22T06:33:17.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.388+0000 7f1556ffd640 1 -- 172.21.15.44:0/2446179521 learned_addr learned my addr 172.21.15.44:0/2446179521 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T06:33:17.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.388+0000 7f15577fe640 1 --2- 172.21.15.44:0/2446179521 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f15581552e0 0x7f155815e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:17.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.388+0000 7f15567fc640 1 --2- 172.21.15.44:0/2446179521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1558154f00 0x7f15580b54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:17.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.389+0000 7f15577fe640 1 -- 172.21.15.44:0/2446179521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1558154f00 msgr2=0x7f15580b54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:17.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.389+0000 7f15577fe640 1 --2- 172.21.15.44:0/2446179521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1558154f00 0x7f15580b54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:17.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.389+0000 7f15577fe640 1 -- 172.21.15.44:0/2446179521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f155806d470 msgr2=0x7f1558159d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:17.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.389+0000 7f15577fe640 1 --2- 172.21.15.44:0/2446179521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f155806d470 0x7f1558159d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:17.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.389+0000 7f15577fe640 1 -- 172.21.15.44:0/2446179521 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15581f1280 con 0x7f15581552e0 2024-04-22T06:33:17.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.389+0000 7f1556ffd640 1 --2- 172.21.15.44:0/2446179521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f155806d470 0x7f1558159d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T06:33:17.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.389+0000 7f15577fe640 1 --2- 172.21.15.44:0/2446179521 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f15581552e0 0x7f155815e1b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1550033000 tx=0x7f1550004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:17.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.389+0000 7f153f7fe640 1 -- 172.21.15.44:0/2446179521 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1550044020 con 0x7f15581552e0 2024-04-22T06:33:17.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.389+0000 7f153f7fe640 1 -- 172.21.15.44:0/2446179521 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f155003f470 con 0x7f15581552e0 2024-04-22T06:33:17.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.390+0000 7f153f7fe640 1 -- 172.21.15.44:0/2446179521 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1550039040 con 0x7f15581552e0 2024-04-22T06:33:17.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.390+0000 7f155d2ba640 1 -- 172.21.15.44:0/2446179521 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15581f1540 con 0x7f15581552e0 2024-04-22T06:33:17.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.390+0000 7f155d2ba640 1 -- 172.21.15.44:0/2446179521 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f15581f1b00 con 0x7f15581552e0 2024-04-22T06:33:17.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.391+0000 7f155d2ba640 1 -- 172.21.15.44:0/2446179521 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1558153b00 con 0x7f15581552e0 2024-04-22T06:33:17.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.391+0000 7f153f7fe640 1 -- 172.21.15.44:0/2446179521 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1550005d10 con 0x7f15581552e0 2024-04-22T06:33:17.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.392+0000 7f153f7fe640 1 --2- 172.21.15.44:0/2446179521 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f154406c920 0x7f154406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:17.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.392+0000 7f153f7fe640 1 -- 172.21.15.44:0/2446179521 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(28..28 src has 1..28) ==== 6821+0+0 (secure 0 0 0) 0x7f15500ae830 con 0x7f15581552e0 2024-04-22T06:33:17.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.392+0000 7f1556ffd640 1 --2- 172.21.15.44:0/2446179521 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f154406c920 0x7f154406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T06:33:17.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.392+0000 7f1556ffd640 1 --2- 172.21.15.44:0/2446179521 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f154406c920 0x7f154406ede0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1548004e60 tx=0x7f1548007f10 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T06:33:17.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.397+0000 7f153f7fe640 1 -- 172.21.15.44:0/2446179521 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1558153b00 con 0x7f15581552e0 2024-04-22T06:33:17.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.595+0000 7f155d2ba640 1 -- 172.21.15.44:0/2446179521 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f15580cdf10 con 0x7f15581552e0 2024-04-22T06:33:17.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.596+0000 7f153f7fe640 1 -- 172.21.15.44:0/2446179521 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v28) ==== 80+0+122 (secure 0 0 0) 0x7f15580cdf10 con 0x7f15581552e0 2024-04-22T06:33:17.594 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6834/1497302315 2024-04-23T06:33:07.139686+0000 2024-04-22T06:33:17.595 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6832/1497302315 2024-04-23T06:33:07.139686+0000 2024-04-22T06:33:17.595 INFO:teuthology.orchestra.run.smithi044.stderr:listed 2 entries 2024-04-22T06:33:17.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.599+0000 7f155d2ba640 1 -- 172.21.15.44:0/2446179521 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f154406c920 msgr2=0x7f154406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:17.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.599+0000 7f155d2ba640 1 --2- 172.21.15.44:0/2446179521 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f154406c920 0x7f154406ede0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1548004e60 tx=0x7f1548007f10 comp rx=0 tx=0).stop 2024-04-22T06:33:17.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.599+0000 7f155d2ba640 1 -- 172.21.15.44:0/2446179521 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f15581552e0 msgr2=0x7f155815e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T06:33:17.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.599+0000 7f155d2ba640 1 --2- 172.21.15.44:0/2446179521 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f15581552e0 0x7f155815e1b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1550033000 tx=0x7f1550004a40 comp rx=0 tx=0).stop 2024-04-22T06:33:17.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.599+0000 7f155d2ba640 1 -- 172.21.15.44:0/2446179521 shutdown_connections 2024-04-22T06:33:17.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.599+0000 7f155d2ba640 1 --2- 172.21.15.44:0/2446179521 >> [v2:172.21.15.70:6832/2570466131,v1:172.21.15.70:6833/2570466131] conn(0x7f154406c920 0x7f154406ede0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:17.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.599+0000 7f155d2ba640 1 --2- 172.21.15.44:0/2446179521 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f15581552e0 0x7f155815e1b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:17.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.599+0000 7f155d2ba640 1 --2- 172.21.15.44:0/2446179521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1558154f00 0x7f15580b54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:17.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.599+0000 7f155d2ba640 1 --2- 172.21.15.44:0/2446179521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f155806d470 0x7f1558159d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:17.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.599+0000 7f155d2ba640 1 -- 172.21.15.44:0/2446179521 >> 172.21.15.44:0/2446179521 conn(0x7f15580733c0 msgr2=0x7f15581533f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T06:33:17.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.599+0000 7f155d2ba640 1 -- 172.21.15.44:0/2446179521 shutdown_connections 2024-04-22T06:33:17.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:17.599+0000 7f155d2ba640 1 -- 172.21.15.44:0/2446179521 wait complete. 2024-04-22T06:33:17.612 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi044.front.sepia.ceph.com... 2024-04-22T06:33:17.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T06:33:17.613 DEBUG:teuthology.orchestra.run.smithi044:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-22T06:33:17.642 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T06:33:17.642 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-04-22T06:33:17.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T06:33:17.642 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-22T06:33:19.209 INFO:tasks.ceph.osd.0:Sent signal 15 2024-04-22T06:33:19.209 INFO:tasks.ceph.osd.1:Sent signal 15 2024-04-22T06:33:19.209 INFO:tasks.ceph.osd.2:Sent signal 15 2024-04-22T06:33:19.210 INFO:tasks.ceph.osd.3:Sent signal 15 2024-04-22T06:33:19.210 INFO:tasks.ceph.osd.4:Sent signal 15 2024-04-22T06:33:19.210 INFO:tasks.ceph.osd.5:Sent signal 15 2024-04-22T06:33:19.211 INFO:tasks.ceph.osd.6:Sent signal 15 2024-04-22T06:33:19.211 INFO:tasks.ceph.osd.7:Sent signal 15 2024-04-22T06:33:19.211 INFO:tasks.ceph.mds.d:Sent signal 15 2024-04-22T06:33:19.211 INFO:tasks.ceph.mds.f:Sent signal 15 2024-04-22T06:33:19.211 INFO:tasks.ceph.mon.a:Sent signal 15 2024-04-22T06:33:19.212 INFO:tasks.ceph.mon.b:Sent signal 15 2024-04-22T06:33:19.212 INFO:tasks.ceph.mon.c:Sent signal 15 2024-04-22T06:33:19.212 INFO:tasks.ceph.mgr.y:Sent signal 15 2024-04-22T06:33:19.212 INFO:tasks.ceph.mgr.x:Sent signal 15 2024-04-22T06:33:19.213 INFO:tasks.ceph.osd.4.smithi070.stderr:2024-04-22T06:33:19.212+0000 7f05930a0640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33684) UID: 0 2024-04-22T06:33:19.213 INFO:tasks.ceph.osd.4.smithi070.stderr:2024-04-22T06:33:19.212+0000 7f05930a0640 -1 osd.4 28 *** Got signal Terminated *** 2024-04-22T06:33:19.213 INFO:tasks.ceph.osd.4.smithi070.stderr:2024-04-22T06:33:19.212+0000 7f05930a0640 -1 osd.4 28 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T06:33:19.214 INFO:tasks.ceph.osd.5.smithi070.stderr:2024-04-22T06:33:19.212+0000 7fb035be3640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33686) UID: 0 2024-04-22T06:33:19.214 INFO:tasks.ceph.osd.5.smithi070.stderr:2024-04-22T06:33:19.212+0000 7fb035be3640 -1 osd.5 28 *** Got signal Terminated *** 2024-04-22T06:33:19.214 INFO:tasks.ceph.osd.5.smithi070.stderr:2024-04-22T06:33:19.212+0000 7fb035be3640 -1 osd.5 28 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T06:33:19.214 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-22T06:33:19.211+0000 7f05112ce640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33436) UID: 0 2024-04-22T06:33:19.214 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-22T06:33:19.211+0000 7f05112ce640 -1 osd.0 28 *** Got signal Terminated *** 2024-04-22T06:33:19.214 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-22T06:33:19.211+0000 7f05112ce640 -1 osd.0 28 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T06:33:19.215 INFO:tasks.ceph.osd.6.smithi070.stderr:2024-04-22T06:33:19.212+0000 7f1fe1787640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33690) UID: 0 2024-04-22T06:33:19.215 INFO:tasks.ceph.osd.6.smithi070.stderr:2024-04-22T06:33:19.212+0000 7f1fe1787640 -1 osd.6 28 *** Got signal Terminated *** 2024-04-22T06:33:19.215 INFO:tasks.ceph.osd.6.smithi070.stderr:2024-04-22T06:33:19.212+0000 7f1fe1787640 -1 osd.6 28 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T06:33:19.215 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-04-22T06:33:19.212+0000 7fa727e14640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33440) UID: 0 2024-04-22T06:33:19.216 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-04-22T06:33:19.212+0000 7fa727e14640 -1 osd.1 28 *** Got signal Terminated *** 2024-04-22T06:33:19.216 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-04-22T06:33:19.212+0000 7fa727e14640 -1 osd.1 28 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T06:33:19.216 INFO:tasks.ceph.osd.7.smithi070.stderr:2024-04-22T06:33:19.213+0000 7ff5f2a27640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33692) UID: 0 2024-04-22T06:33:19.216 INFO:tasks.ceph.osd.7.smithi070.stderr:2024-04-22T06:33:19.213+0000 7ff5f2a27640 -1 osd.7 28 *** Got signal Terminated *** 2024-04-22T06:33:19.216 INFO:tasks.ceph.osd.7.smithi070.stderr:2024-04-22T06:33:19.213+0000 7ff5f2a27640 -1 osd.7 28 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T06:33:19.217 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-04-22T06:33:19.212+0000 7f4c4a292640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33442) UID: 0 2024-04-22T06:33:19.217 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-04-22T06:33:19.212+0000 7f4c4a292640 -1 osd.2 28 *** Got signal Terminated *** 2024-04-22T06:33:19.217 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-04-22T06:33:19.212+0000 7f4c4a292640 -1 osd.2 28 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T06:33:19.217 INFO:tasks.ceph.mds.d.smithi070.stderr:2024-04-22T06:33:19.213+0000 7fd91a9ad640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i d (PID: 39566) UID: 0 2024-04-22T06:33:19.217 INFO:tasks.ceph.mds.d.smithi070.stderr:2024-04-22T06:33:19.213+0000 7fd91a9ad640 -1 mds.d *** got signal Terminated *** 2024-04-22T06:33:19.218 INFO:tasks.ceph.mds.f.smithi070.stderr:2024-04-22T06:33:19.213+0000 7f7926cfa640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i f (PID: 39568) UID: 0 2024-04-22T06:33:19.218 INFO:tasks.ceph.mds.f.smithi070.stderr:2024-04-22T06:33:19.213+0000 7f7926cfa640 -1 mds.f *** got signal Terminated *** 2024-04-22T06:33:19.218 INFO:tasks.ceph.osd.3.smithi044.stderr:2024-04-22T06:33:19.212+0000 7f255b9c7640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33445) UID: 0 2024-04-22T06:33:19.218 INFO:tasks.ceph.osd.3.smithi044.stderr:2024-04-22T06:33:19.212+0000 7f255b9c7640 -1 osd.3 28 *** Got signal Terminated *** 2024-04-22T06:33:19.218 INFO:tasks.ceph.osd.3.smithi044.stderr:2024-04-22T06:33:19.212+0000 7f255b9c7640 -1 osd.3 28 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T06:33:19.218 INFO:tasks.ceph.mon.b.smithi070.stderr:2024-04-22T06:33:19.214+0000 7f79e9b4f640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i b (PID: 33105) UID: 0 2024-04-22T06:33:19.219 INFO:tasks.ceph.mon.b.smithi070.stderr:2024-04-22T06:33:19.214+0000 7f79e9b4f640 -1 mon.b@1(peon) e1 *** Got Signal Terminated *** 2024-04-22T06:33:19.219 INFO:tasks.ceph.mon.a.smithi044.stderr:2024-04-22T06:33:19.214+0000 7f5b02ed6640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i a (PID: 33163) UID: 0 2024-04-22T06:33:19.219 INFO:tasks.ceph.mon.a.smithi044.stderr:2024-04-22T06:33:19.214+0000 7f5b02ed6640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-22T06:33:19.219 INFO:tasks.ceph.mon.c.smithi070.stderr:2024-04-22T06:33:19.214+0000 7fa9d6fad640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i c (PID: 33107) UID: 0 2024-04-22T06:33:19.219 INFO:tasks.ceph.mon.c.smithi070.stderr:2024-04-22T06:33:19.214+0000 7fa9d6fad640 -1 mon.c@2(peon) e1 *** Got Signal Terminated *** 2024-04-22T06:33:19.413 INFO:tasks.ceph.mgr.y.smithi044.stderr:daemon-helper: command crashed with signal 15 2024-04-22T06:33:38.899 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T06:33:38.902 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T06:33:39.863 INFO:tasks.workunit.client.0.smithi044.stderr:Updating files: 94% (11553/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-22T06:33:40.715 INFO:tasks.workunit.client.0.smithi044.stderr:Note: switching to '86f7587a5a09af35f0895e1a2d08527638fae697'. 2024-04-22T06:33:40.715 INFO:tasks.workunit.client.0.smithi044.stderr: 2024-04-22T06:33:40.715 INFO:tasks.workunit.client.0.smithi044.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-22T06:33:40.716 INFO:tasks.workunit.client.0.smithi044.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-22T06:33:40.716 INFO:tasks.workunit.client.0.smithi044.stderr:state without impacting any branches by switching back to a branch. 2024-04-22T06:33:40.716 INFO:tasks.workunit.client.0.smithi044.stderr: 2024-04-22T06:33:40.716 INFO:tasks.workunit.client.0.smithi044.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-22T06:33:40.716 INFO:tasks.workunit.client.0.smithi044.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-22T06:33:40.716 INFO:tasks.workunit.client.0.smithi044.stderr: 2024-04-22T06:33:40.716 INFO:tasks.workunit.client.0.smithi044.stderr: git switch -c 2024-04-22T06:33:40.716 INFO:tasks.workunit.client.0.smithi044.stderr: 2024-04-22T06:33:40.716 INFO:tasks.workunit.client.0.smithi044.stderr:Or undo this operation with: 2024-04-22T06:33:40.716 INFO:tasks.workunit.client.0.smithi044.stderr: 2024-04-22T06:33:40.716 INFO:tasks.workunit.client.0.smithi044.stderr: git switch - 2024-04-22T06:33:40.716 INFO:tasks.workunit.client.0.smithi044.stderr: 2024-04-22T06:33:40.716 INFO:tasks.workunit.client.0.smithi044.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-22T06:33:40.717 INFO:tasks.workunit.client.0.smithi044.stderr: 2024-04-22T06:33:40.717 INFO:tasks.workunit.client.0.smithi044.stderr:HEAD is now at 86f7587a5a0 qa/fsx: use the master branch to build the xfstest-dev 2024-04-22T06:33:40.722 DEBUG:teuthology.orchestra.run.smithi044:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-22T06:33:40.750 INFO:tasks.workunit.client.0.smithi044.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-22T06:33:40.752 INFO:tasks.workunit.client.0.smithi044.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-22T06:33:40.752 INFO:tasks.workunit.client.0.smithi044.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-22T06:33:41.348 INFO:tasks.workunit.client.0.smithi044.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-22T06:33:41.407 INFO:tasks.workunit.client.0.smithi044.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-22T06:33:41.449 INFO:tasks.workunit.client.0.smithi044.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-22T06:33:41.450 INFO:tasks.workunit.client.0.smithi044.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-22T06:33:41.450 INFO:tasks.workunit.client.0.smithi044.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-22T06:33:41.490 INFO:tasks.workunit.client.0.smithi044.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-22T06:33:41.494 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T06:33:41.494 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-22T06:33:41.565 INFO:tasks.workunit:Running workunits matching fs/misc/trivial_sync.sh on client.0... 2024-04-22T06:33:41.566 INFO:tasks.workunit:Running workunit fs/misc/trivial_sync.sh... 2024-04-22T06:33:41.567 DEBUG:teuthology.orchestra.run.smithi044:workunit test fs/misc/trivial_sync.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=86f7587a5a09af35f0895e1a2d08527638fae697 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/trivial_sync.sh 2024-04-22T06:33:43.595 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-22T06:33:43.595 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-22T06:33:43.626 INFO:tasks.workunit:Stopping ['fs/misc/trivial_sync.sh'] on client.0... 2024-04-22T06:33:43.626 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-22T06:33:43.959 DEBUG:teuthology.parallel:result is None 2024-04-22T06:33:43.960 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-22T06:33:43.984 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-22T06:33:43.984 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-04-22T06:33:43.993 INFO:tasks.quiescer:joining Quiescers 2024-04-22T06:33:43.993 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T06:33:44.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:44.126+0000 7faca7443640 1 Processor -- start 2024-04-22T06:33:44.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:44.126+0000 7faca7443640 1 -- start start 2024-04-22T06:33:44.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:44.127+0000 7faca7443640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7faca00c9ae0 0x7faca0073960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:44.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:44.127+0000 7faca7443640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca015b070 con 0x7faca01552b0 2024-04-22T06:33:44.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:44.127+0000 7faca7443640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca015f160 con 0x7faca015be10 2024-04-22T06:33:44.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:44.127+0000 7faca7443640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7faca015b980 con 0x7faca00c9ae0 2024-04-22T06:33:44.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:44.127+0000 7faca51df640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7faca01552b0 legacy=0x7faca00ca260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:33:44.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:44.127+0000 7faca59e0640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7faca00c9ae0 msgr2=0x7faca0073960 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:33:44.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:44.127+0000 7faca59e0640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7faca00c9ae0 0x7faca0073960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:33:44.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:44.127+0000 7faca49de640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7faca015be10 legacy=0x7faca015e660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:33:44.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:44.328+0000 7faca51df640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7faca01552b0 legacy=0x7faca00ca260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:33:44.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:44.328+0000 7faca59e0640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7faca00c9ae0 msgr2=0x7faca0073960 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:33:44.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:44.328+0000 7faca59e0640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7faca00c9ae0 0x7faca0073960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:33:44.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:44.328+0000 7faca49de640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7faca015be10 legacy=0x7faca015e660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:33:44.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:44.728+0000 7faca51df640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7faca01552b0 legacy=0x7faca00ca260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:33:44.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:44.728+0000 7faca59e0640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7faca00c9ae0 msgr2=0x7faca0073960 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:33:44.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:44.728+0000 7faca59e0640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7faca00c9ae0 0x7faca0073960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:33:44.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:44.728+0000 7faca49de640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7faca015be10 legacy=0x7faca015e660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:33:45.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:45.529+0000 7faca51df640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7faca01552b0 legacy=0x7faca00ca260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:33:45.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:45.529+0000 7faca59e0640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7faca00c9ae0 msgr2=0x7faca0073960 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:33:45.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:45.529+0000 7faca59e0640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7faca00c9ae0 0x7faca0073960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:33:45.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:45.529+0000 7faca49de640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7faca015be10 legacy=0x7faca015e660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:33:47.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:47.127+0000 7fac96ffd640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7faca00c9ae0 msgr2=0x7faca0073960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:33:47.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:47.127+0000 7fac96ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7faca00c9ae0 0x7faca0073960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:47.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:47.127+0000 7fac96ffd640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7faca015be10 legacy=0x7faca015e660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:33:47.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:47.127+0000 7fac96ffd640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7faca01552b0 legacy=0x7faca00ca260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:33:47.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:47.127+0000 7fac96ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac840036a0 0x7fac84005b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:47.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:47.127+0000 7fac96ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006110 0x7fac840085d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:47.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:47.127+0000 7fac96ffd640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7faca015b980 con 0x7fac840036a0 2024-04-22T06:33:47.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:47.127+0000 7fac96ffd640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fac84008ca0 con 0x7fac84006110 2024-04-22T06:33:47.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:47.127+0000 7faca51df640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac840036a0 msgr2=0x7fac84005b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:33:47.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:47.127+0000 7faca51df640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac840036a0 0x7fac84005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:33:47.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:47.127+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006110 msgr2=0x7fac840085d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:33:47.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:47.127+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006110 0x7fac840085d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:33:47.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:47.328+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006110 msgr2=0x7fac840085d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:33:47.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:47.328+0000 7faca51df640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac840036a0 msgr2=0x7fac84005b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:33:47.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:47.328+0000 7faca51df640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac840036a0 0x7fac84005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:33:47.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:47.328+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006110 0x7fac840085d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:33:47.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:47.729+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006110 msgr2=0x7fac840085d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:33:47.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:47.729+0000 7faca51df640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac840036a0 msgr2=0x7fac84005b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:33:47.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:47.729+0000 7faca51df640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac840036a0 0x7fac84005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:33:47.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:47.729+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006110 0x7fac840085d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:33:48.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:48.530+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006110 msgr2=0x7fac840085d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:33:48.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:48.530+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006110 0x7fac840085d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:33:48.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:48.530+0000 7faca51df640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac840036a0 msgr2=0x7fac84005b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:33:48.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:48.530+0000 7faca51df640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac840036a0 0x7fac84005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:33:50.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:50.127+0000 7fac96ffd640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006110 msgr2=0x7fac840085d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:33:50.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:50.127+0000 7fac96ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006110 0x7fac840085d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:50.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:50.127+0000 7fac96ffd640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac840036a0 msgr2=0x7fac84005b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:33:50.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:50.127+0000 7fac96ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac840036a0 0x7fac84005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:50.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:50.127+0000 7faca59e0640 1 -- reap_dead start 2024-04-22T06:33:50.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:50.127+0000 7fac96ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006110 0x7fac840085d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:50.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:50.127+0000 7fac96ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac8400a720 0x7fac8400cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:50.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:50.127+0000 7fac96ffd640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca015f440 con 0x7fac84003ab0 2024-04-22T06:33:50.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:50.127+0000 7fac96ffd640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7faca015b980 con 0x7fac8400a720 2024-04-22T06:33:50.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:50.127+0000 7fac96ffd640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fac84008ca0 con 0x7fac84006110 2024-04-22T06:33:50.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:50.127+0000 7faca59e0640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac8400a720 msgr2=0x7fac8400cb10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:33:50.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:50.127+0000 7faca49de640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84003ab0 legacy=0x7fac84009000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:33:50.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:50.127+0000 7faca59e0640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac8400a720 0x7fac8400cb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:33:50.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:50.127+0000 7faca51df640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006110 msgr2=0x7fac840085d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:33:50.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:50.127+0000 7faca51df640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006110 0x7fac840085d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:33:50.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:50.328+0000 7faca59e0640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac8400a720 msgr2=0x7fac8400cb10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:33:50.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:50.328+0000 7faca49de640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84003ab0 legacy=0x7fac84009000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:33:50.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:50.328+0000 7faca59e0640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac8400a720 0x7fac8400cb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:33:50.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:50.328+0000 7faca51df640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006110 msgr2=0x7fac840085d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:33:50.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:50.328+0000 7faca51df640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006110 0x7fac840085d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:33:50.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:50.729+0000 7faca49de640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84003ab0 legacy=0x7fac84009000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:33:50.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:50.729+0000 7faca59e0640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac8400a720 msgr2=0x7fac8400cb10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:33:50.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:50.729+0000 7faca59e0640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac8400a720 0x7fac8400cb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:33:50.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:50.729+0000 7faca51df640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006110 msgr2=0x7fac840085d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:33:50.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:50.729+0000 7faca51df640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006110 0x7fac840085d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:33:51.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:51.530+0000 7faca59e0640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac8400a720 msgr2=0x7fac8400cb10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:33:51.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:51.530+0000 7faca49de640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84003ab0 legacy=0x7fac84009000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:33:51.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:51.530+0000 7faca59e0640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac8400a720 0x7fac8400cb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:33:51.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:51.530+0000 7faca51df640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006110 msgr2=0x7fac840085d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:33:51.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:51.530+0000 7faca51df640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006110 0x7fac840085d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:33:53.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:53.128+0000 7fac96ffd640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006110 msgr2=0x7fac840085d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:33:53.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:53.128+0000 7fac96ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006110 0x7fac840085d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:53.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:53.128+0000 7fac96ffd640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac8400a720 msgr2=0x7fac8400cb10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:33:53.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:53.128+0000 7fac96ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac8400a720 0x7fac8400cb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:53.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:53.128+0000 7fac96ffd640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84003ab0 legacy=0x7fac84009000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:33:53.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:53.128+0000 7fac96ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac840113e0 0x7fac840137d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:53.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:53.128+0000 7fac96ffd640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca015b070 con 0x7fac8400d2a0 2024-04-22T06:33:53.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:53.128+0000 7fac96ffd640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7faca015b980 con 0x7fac840113e0 2024-04-22T06:33:53.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:53.128+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac840113e0 msgr2=0x7fac840137d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:33:53.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:53.128+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac8400d2a0 legacy=0x7fac8400d6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:33:53.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:53.128+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac840113e0 0x7fac840137d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:33:53.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:53.329+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac8400d2a0 legacy=0x7fac8400d6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:33:53.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:53.329+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac840113e0 msgr2=0x7fac840137d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:33:53.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:53.329+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac840113e0 0x7fac840137d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:33:53.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:53.730+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac8400d2a0 legacy=0x7fac8400d6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:33:53.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:53.730+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac840113e0 msgr2=0x7fac840137d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:33:53.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:53.730+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac840113e0 0x7fac840137d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:33:54.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:54.531+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac8400d2a0 legacy=0x7fac8400d6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:33:54.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:54.531+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac840113e0 msgr2=0x7fac840137d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:33:54.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:54.531+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac840113e0 0x7fac840137d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:33:56.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:56.128+0000 7fac96ffd640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac840113e0 msgr2=0x7fac840137d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:33:56.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:56.128+0000 7fac96ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac840113e0 0x7fac840137d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:56.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:56.128+0000 7fac96ffd640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac8400d2a0 legacy=0x7fac8400d6b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:33:56.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:56.128+0000 7faca59e0640 1 -- reap_dead start 2024-04-22T06:33:56.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:56.128+0000 7fac96ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac840113e0 0x7fac840137d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:56.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:56.128+0000 7fac96ffd640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca015f440 con 0x7fac84003ab0 2024-04-22T06:33:56.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:56.128+0000 7fac96ffd640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca015b070 con 0x7fac84013d10 2024-04-22T06:33:56.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:56.128+0000 7fac96ffd640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7faca015b980 con 0x7fac840113e0 2024-04-22T06:33:56.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:56.128+0000 7faca59e0640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84003ab0 legacy=0x7fac84009000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:33:56.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:56.128+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac840113e0 msgr2=0x7fac840137d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:33:56.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:56.128+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84013d10 legacy=0x7fac84016100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:33:56.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:56.128+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac840113e0 0x7fac840137d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:33:56.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:56.329+0000 7faca59e0640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84003ab0 legacy=0x7fac84009000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:33:56.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:56.329+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac840113e0 msgr2=0x7fac840137d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:33:56.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:56.329+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84013d10 legacy=0x7fac84016100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:33:56.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:56.329+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac840113e0 0x7fac840137d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:33:56.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:56.730+0000 7faca59e0640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84003ab0 legacy=0x7fac84009000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:33:56.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:56.730+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84013d10 legacy=0x7fac84016100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:33:56.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:56.730+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac840113e0 msgr2=0x7fac840137d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:33:56.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:56.730+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac840113e0 0x7fac840137d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:33:57.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:57.531+0000 7faca59e0640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84003ab0 legacy=0x7fac84009000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:33:57.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:57.531+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84013d10 legacy=0x7fac84016100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:33:57.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:57.531+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac840113e0 msgr2=0x7fac840137d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:33:57.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:57.531+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac840113e0 0x7fac840137d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:33:59.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:59.128+0000 7fac96ffd640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac840113e0 msgr2=0x7fac840137d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:33:59.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:59.128+0000 7fac96ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac840113e0 0x7fac840137d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:33:59.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:59.128+0000 7fac96ffd640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84013d10 legacy=0x7fac84016100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:33:59.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:59.128+0000 7fac96ffd640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84003ab0 legacy=0x7fac84009000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:33:59.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:59.128+0000 7fac96ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400d710 0x7fac840085d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:59.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:59.128+0000 7fac96ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac8400ab00 0x7fac8400e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:33:59.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:59.128+0000 7fac96ffd640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7faca015b980 con 0x7fac8400ab00 2024-04-22T06:33:59.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:59.128+0000 7fac96ffd640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fac84008ca0 con 0x7fac8400d710 2024-04-22T06:33:59.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:59.128+0000 7faca49de640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac8400ab00 msgr2=0x7fac8400e3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:33:59.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:59.128+0000 7faca49de640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac8400ab00 0x7fac8400e3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:33:59.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:59.128+0000 7faca51df640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400d710 msgr2=0x7fac840085d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:33:59.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:59.129+0000 7faca51df640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400d710 0x7fac840085d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:33:59.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:59.329+0000 7faca49de640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac8400ab00 msgr2=0x7fac8400e3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:33:59.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:59.329+0000 7faca49de640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac8400ab00 0x7fac8400e3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:33:59.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:59.330+0000 7faca51df640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400d710 msgr2=0x7fac840085d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:33:59.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:59.330+0000 7faca51df640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400d710 0x7fac840085d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:33:59.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:59.730+0000 7faca49de640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac8400ab00 msgr2=0x7fac8400e3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:33:59.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:59.730+0000 7faca49de640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac8400ab00 0x7fac8400e3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:33:59.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:59.731+0000 7faca51df640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400d710 msgr2=0x7fac840085d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:33:59.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:33:59.731+0000 7faca51df640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400d710 0x7fac840085d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:34:00.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:00.531+0000 7faca49de640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac8400ab00 msgr2=0x7fac8400e3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:00.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:00.531+0000 7faca49de640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac8400ab00 0x7fac8400e3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:34:00.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:00.532+0000 7faca51df640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400d710 msgr2=0x7fac840085d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:00.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:00.532+0000 7faca51df640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400d710 0x7fac840085d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:34:02.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:02.128+0000 7fac96ffd640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400d710 msgr2=0x7fac840085d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:02.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:02.128+0000 7fac96ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400d710 0x7fac840085d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:34:02.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:02.128+0000 7fac96ffd640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac8400ab00 msgr2=0x7fac8400e3c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:02.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:02.128+0000 7fac96ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac8400ab00 0x7fac8400e3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:34:02.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:02.129+0000 7faca59e0640 1 -- reap_dead start 2024-04-22T06:34:02.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:02.129+0000 7fac96ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d710 0x7fac84007ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:34:02.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:02.129+0000 7fac96ffd640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca015f160 con 0x7fac8400ab00 2024-04-22T06:34:02.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:02.129+0000 7fac96ffd640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca015f440 con 0x7fac84003ab0 2024-04-22T06:34:02.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:02.129+0000 7fac96ffd640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7faca015b980 con 0x7fac8400d710 2024-04-22T06:34:02.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:02.129+0000 7faca49de640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac8400ab00 legacy=0x7fac840065c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:34:02.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:02.129+0000 7faca59e0640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d710 msgr2=0x7fac84007ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:34:02.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:02.129+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84003ab0 legacy=0x7fac84008fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:34:02.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:02.129+0000 7faca59e0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d710 0x7fac84007ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:34:02.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:02.330+0000 7faca49de640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac8400ab00 legacy=0x7fac840065c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:34:02.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:02.330+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84003ab0 legacy=0x7fac84008fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:34:02.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:02.330+0000 7faca59e0640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d710 msgr2=0x7fac84007ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:34:02.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:02.330+0000 7faca59e0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d710 0x7fac84007ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:34:02.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:02.731+0000 7faca49de640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac8400ab00 legacy=0x7fac840065c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:34:02.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:02.731+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84003ab0 legacy=0x7fac84008fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:34:02.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:02.731+0000 7faca59e0640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d710 msgr2=0x7fac84007ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:34:02.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:02.731+0000 7faca59e0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d710 0x7fac84007ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:34:03.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:03.532+0000 7faca49de640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac8400ab00 legacy=0x7fac840065c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:34:03.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:03.532+0000 7faca59e0640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d710 msgr2=0x7fac84007ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:34:03.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:03.532+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84003ab0 legacy=0x7fac84008fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:34:03.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:03.532+0000 7faca59e0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d710 0x7fac84007ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:34:05.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:05.129+0000 7fac96ffd640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d710 msgr2=0x7fac84007ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:05.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:05.129+0000 7fac96ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d710 0x7fac84007ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:34:05.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:05.129+0000 7fac96ffd640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84003ab0 legacy=0x7fac84008fc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:05.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:05.129+0000 7fac96ffd640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac8400ab00 legacy=0x7fac840065c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:05.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:05.129+0000 7fac96ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84008220 0x7fac84008600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:34:05.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:05.129+0000 7fac96ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac840123c0 0x7fac840127d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:34:05.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:05.129+0000 7fac96ffd640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7faca015b980 con 0x7fac840123c0 2024-04-22T06:34:05.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:05.129+0000 7fac96ffd640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fac84008ca0 con 0x7fac84008220 2024-04-22T06:34:05.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:05.129+0000 7faca49de640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac840123c0 msgr2=0x7fac840127d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:05.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:05.129+0000 7faca49de640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac840123c0 0x7fac840127d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:34:05.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:05.129+0000 7faca51df640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84008220 msgr2=0x7fac84008600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:05.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:05.129+0000 7faca51df640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84008220 0x7fac84008600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:34:05.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:05.330+0000 7faca49de640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac840123c0 msgr2=0x7fac840127d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:05.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:05.330+0000 7faca49de640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac840123c0 0x7fac840127d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:34:05.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:05.330+0000 7faca51df640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84008220 msgr2=0x7fac84008600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:05.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:05.330+0000 7faca51df640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84008220 0x7fac84008600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:34:05.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:05.731+0000 7faca49de640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac840123c0 msgr2=0x7fac840127d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:05.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:05.731+0000 7faca49de640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac840123c0 0x7fac840127d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:34:05.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:05.731+0000 7faca51df640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84008220 msgr2=0x7fac84008600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:05.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:05.731+0000 7faca51df640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84008220 0x7fac84008600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:34:06.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:06.532+0000 7faca49de640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac840123c0 msgr2=0x7fac840127d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:06.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:06.532+0000 7faca49de640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac840123c0 0x7fac840127d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:34:06.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:06.532+0000 7faca51df640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84008220 msgr2=0x7fac84008600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:06.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:06.532+0000 7faca51df640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84008220 0x7fac84008600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:34:08.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:08.129+0000 7fac96ffd640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84008220 msgr2=0x7fac84008600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:08.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:08.129+0000 7fac96ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84008220 0x7fac84008600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:34:08.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:08.129+0000 7fac96ffd640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac840123c0 msgr2=0x7fac840127d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:08.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:08.129+0000 7fac96ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac840123c0 0x7fac840127d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:34:08.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:08.129+0000 7faca59e0640 1 -- reap_dead start 2024-04-22T06:34:08.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:08.129+0000 7fac96ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d710 0x7fac84007ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:34:08.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:08.129+0000 7fac96ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac840127d0 0x7fac84008220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:34:08.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:08.129+0000 7fac96ffd640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca015b070 con 0x7fac8400cb10 2024-04-22T06:34:08.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:08.129+0000 7fac96ffd640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7faca015b980 con 0x7fac8400d710 2024-04-22T06:34:08.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:08.129+0000 7fac96ffd640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fac84008ca0 con 0x7fac840127d0 2024-04-22T06:34:08.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:08.130+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac840127d0 msgr2=0x7fac84008220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:08.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:08.130+0000 7faca59e0640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac8400cb10 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:34:08.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:08.130+0000 7faca51df640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d710 msgr2=0x7fac84007ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:34:08.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:08.130+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac840127d0 0x7fac84008220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:34:08.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:08.130+0000 7faca51df640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d710 0x7fac84007ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:34:08.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:08.331+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac840127d0 msgr2=0x7fac84008220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:08.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:08.331+0000 7faca51df640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d710 msgr2=0x7fac84007ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:34:08.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:08.331+0000 7faca59e0640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac8400cb10 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:34:08.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:08.331+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac840127d0 0x7fac84008220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:34:08.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:08.331+0000 7faca51df640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d710 0x7fac84007ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:34:08.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:08.732+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac840127d0 msgr2=0x7fac84008220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:08.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:08.732+0000 7faca59e0640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac8400cb10 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:34:08.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:08.732+0000 7faca51df640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d710 msgr2=0x7fac84007ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:34:08.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:08.732+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac840127d0 0x7fac84008220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:34:08.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:08.732+0000 7faca51df640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d710 0x7fac84007ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:34:08.960 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T06:34:08.963 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T06:34:09.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:09.533+0000 7faca51df640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d710 msgr2=0x7fac84007ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:34:09.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:09.533+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac840127d0 msgr2=0x7fac84008220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:09.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:09.533+0000 7faca59e0640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac8400cb10 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:34:09.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:09.533+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac840127d0 0x7fac84008220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:34:09.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:09.533+0000 7faca51df640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d710 0x7fac84007ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:34:11.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:11.130+0000 7fac96ffd640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac840127d0 msgr2=0x7fac84008220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:11.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:11.130+0000 7fac96ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac840127d0 0x7fac84008220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:34:11.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:11.130+0000 7fac96ffd640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d710 msgr2=0x7fac84007ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:11.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:11.130+0000 7fac96ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d710 0x7fac84007ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:34:11.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:11.130+0000 7fac96ffd640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac8400cb10 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:11.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:11.130+0000 7fac96ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac840123c0 0x7fac840075b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:34:11.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:11.130+0000 7fac96ffd640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca015f160 con 0x7fac84008760 2024-04-22T06:34:11.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:11.130+0000 7fac96ffd640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7faca015b980 con 0x7fac840123c0 2024-04-22T06:34:11.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:11.130+0000 7faca51df640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac840123c0 msgr2=0x7fac840075b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:11.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:11.130+0000 7faca49de640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84008760 legacy=0x7fac840135d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:34:11.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:11.130+0000 7faca51df640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac840123c0 0x7fac840075b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:34:11.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:11.331+0000 7faca51df640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac840123c0 msgr2=0x7fac840075b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:11.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:11.331+0000 7faca49de640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84008760 legacy=0x7fac840135d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:34:11.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:11.331+0000 7faca51df640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac840123c0 0x7fac840075b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:34:11.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:11.732+0000 7faca51df640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac840123c0 msgr2=0x7fac840075b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:11.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:11.732+0000 7faca49de640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84008760 legacy=0x7fac840135d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:34:11.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:11.732+0000 7faca51df640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac840123c0 0x7fac840075b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:34:12.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:12.533+0000 7faca49de640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84008760 legacy=0x7fac840135d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:34:12.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:12.533+0000 7faca51df640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac840123c0 msgr2=0x7fac840075b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:12.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:12.533+0000 7faca51df640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac840123c0 0x7fac840075b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:34:14.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:14.130+0000 7fac96ffd640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac840123c0 msgr2=0x7fac840075b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:14.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:14.130+0000 7fac96ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac840123c0 0x7fac840075b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:34:14.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:14.130+0000 7fac96ffd640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84008760 legacy=0x7fac840135d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:14.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:14.130+0000 7faca59e0640 1 -- reap_dead start 2024-04-22T06:34:14.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:14.130+0000 7fac96ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac840127d0 0x7fac84012ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:34:14.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:14.130+0000 7fac96ffd640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca015b070 con 0x7fac84005a90 2024-04-22T06:34:14.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:14.130+0000 7fac96ffd640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca015f160 con 0x7fac8400d710 2024-04-22T06:34:14.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:14.130+0000 7fac96ffd640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7faca015b980 con 0x7fac840127d0 2024-04-22T06:34:14.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:14.130+0000 7faca51df640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84005a90 legacy=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:34:14.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:14.130+0000 7faca59e0640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac840127d0 msgr2=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:14.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:14.130+0000 7faca59e0640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac840127d0 0x7fac84012ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:34:14.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:14.130+0000 7faca49de640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac8400d710 legacy=0x7fac84007c30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:34:14.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:14.331+0000 7faca59e0640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac840127d0 msgr2=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:14.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:14.331+0000 7faca51df640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84005a90 legacy=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:34:14.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:14.331+0000 7faca59e0640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac840127d0 0x7fac84012ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:34:14.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:14.331+0000 7faca49de640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac8400d710 legacy=0x7fac84007c30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:34:14.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:14.732+0000 7faca59e0640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac840127d0 msgr2=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:14.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:14.732+0000 7faca51df640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84005a90 legacy=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:34:14.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:14.732+0000 7faca59e0640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac840127d0 0x7fac84012ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:34:14.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:14.732+0000 7faca49de640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac8400d710 legacy=0x7fac84007c30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:34:15.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:15.533+0000 7faca51df640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84005a90 legacy=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:34:15.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:15.533+0000 7faca59e0640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac840127d0 msgr2=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:15.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:15.533+0000 7faca59e0640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac840127d0 0x7fac84012ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:34:15.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:15.533+0000 7faca49de640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac8400d710 legacy=0x7fac84007c30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:34:17.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:17.131+0000 7fac96ffd640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac840127d0 msgr2=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:17.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:17.131+0000 7fac96ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac840127d0 0x7fac84012ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:34:17.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:17.131+0000 7fac96ffd640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac8400d710 legacy=0x7fac84007c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:17.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:17.131+0000 7fac96ffd640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84005a90 legacy=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:17.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:17.131+0000 7fac96ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84008340 0x7fac84013400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:34:17.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:17.131+0000 7fac96ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 0x7fac840065a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:34:17.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:17.131+0000 7fac96ffd640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7faca015b980 con 0x7fac8400cb10 2024-04-22T06:34:17.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:17.131+0000 7fac96ffd640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fac84008ca0 con 0x7fac84008340 2024-04-22T06:34:17.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:17.131+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 msgr2=0x7fac840065a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:34:17.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:17.131+0000 7faca51df640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84008340 msgr2=0x7fac84013400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:17.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:17.131+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 0x7fac840065a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:34:17.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:17.131+0000 7faca51df640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84008340 0x7fac84013400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:34:17.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:17.332+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 msgr2=0x7fac840065a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:34:17.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:17.332+0000 7faca51df640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84008340 msgr2=0x7fac84013400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:17.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:17.332+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 0x7fac840065a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:34:17.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:17.332+0000 7faca51df640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84008340 0x7fac84013400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:34:17.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:17.733+0000 7faca51df640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84008340 msgr2=0x7fac84013400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:17.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:17.733+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 msgr2=0x7fac840065a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:34:17.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:17.733+0000 7faca51df640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84008340 0x7fac84013400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:34:17.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:17.733+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 0x7fac840065a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:34:18.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:18.534+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 msgr2=0x7fac840065a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:34:18.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:18.534+0000 7faca51df640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84008340 msgr2=0x7fac84013400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:18.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:18.534+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 0x7fac840065a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:34:18.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:18.534+0000 7faca51df640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84008340 0x7fac84013400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:34:20.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:20.131+0000 7fac96ffd640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84008340 msgr2=0x7fac84013400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:20.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:20.131+0000 7fac96ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84008340 0x7fac84013400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:34:20.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:20.131+0000 7fac96ffd640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 msgr2=0x7fac840065a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:20.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:20.131+0000 7fac96ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 0x7fac840065a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:34:20.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:20.131+0000 7faca59e0640 1 -- reap_dead start 2024-04-22T06:34:20.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:20.131+0000 7fac96ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d710 0x7fac840065a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:34:20.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:20.131+0000 7fac96ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84012360 0x7fac84012ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:34:20.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:20.131+0000 7fac96ffd640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca015f440 con 0x7fac84005a90 2024-04-22T06:34:20.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:20.131+0000 7fac96ffd640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7faca015b980 con 0x7fac84012360 2024-04-22T06:34:20.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:20.131+0000 7fac96ffd640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fac84008ca0 con 0x7fac8400d710 2024-04-22T06:34:20.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:20.131+0000 7faca51df640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84005a90 legacy=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:34:20.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:20.131+0000 7faca59e0640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84012360 msgr2=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:20.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:20.131+0000 7faca59e0640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84012360 0x7fac84012ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:34:20.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:20.131+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d710 msgr2=0x7fac840065a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:34:20.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:20.131+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d710 0x7fac840065a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:34:20.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:20.332+0000 7faca51df640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84005a90 legacy=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:34:20.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:20.332+0000 7faca59e0640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84012360 msgr2=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:20.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:20.332+0000 7faca59e0640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84012360 0x7fac84012ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:34:20.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:20.332+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d710 msgr2=0x7fac840065a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:34:20.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:20.332+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d710 0x7fac840065a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:34:20.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:20.733+0000 7faca51df640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84005a90 legacy=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:34:20.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:20.733+0000 7faca59e0640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84012360 msgr2=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:20.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:20.733+0000 7faca59e0640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84012360 0x7fac84012ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:34:20.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:20.733+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d710 msgr2=0x7fac840065a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:34:20.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:20.733+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d710 0x7fac840065a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:34:21.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:21.534+0000 7faca51df640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84005a90 legacy=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:34:21.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:21.534+0000 7faca59e0640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84012360 msgr2=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:21.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:21.534+0000 7faca59e0640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84012360 0x7fac84012ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:34:21.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:21.534+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d710 msgr2=0x7fac840065a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:34:21.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:21.534+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d710 0x7fac840065a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:34:23.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:23.131+0000 7fac96ffd640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d710 msgr2=0x7fac840065a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:23.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:23.131+0000 7fac96ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d710 0x7fac840065a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:34:23.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:23.131+0000 7fac96ffd640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84012360 msgr2=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:23.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:23.131+0000 7fac96ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84012360 0x7fac84012ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:34:23.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:23.131+0000 7fac96ffd640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84005a90 legacy=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:23.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:23.131+0000 7fac96ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400cb10 0x7fac84013400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:34:23.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:23.131+0000 7fac96ffd640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca015b070 con 0x7fac84012740 2024-04-22T06:34:23.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:23.131+0000 7fac96ffd640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7faca015b980 con 0x7fac8400cb10 2024-04-22T06:34:23.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:23.131+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84012740 legacy=0x7fac84007b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:34:23.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:23.131+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400cb10 msgr2=0x7fac84013400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:23.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:23.131+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400cb10 0x7fac84013400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:34:23.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:23.332+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400cb10 msgr2=0x7fac84013400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:23.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:23.332+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84012740 legacy=0x7fac84007b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:34:23.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:23.332+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400cb10 0x7fac84013400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:34:23.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:23.733+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400cb10 msgr2=0x7fac84013400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:23.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:23.733+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84012740 legacy=0x7fac84007b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:34:23.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:23.733+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400cb10 0x7fac84013400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:34:24.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:24.534+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84012740 legacy=0x7fac84007b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:34:24.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:24.534+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400cb10 msgr2=0x7fac84013400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:24.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:24.534+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400cb10 0x7fac84013400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:34:26.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:26.132+0000 7fac96ffd640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400cb10 msgr2=0x7fac84013400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:26.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:26.132+0000 7fac96ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400cb10 0x7fac84013400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:34:26.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:26.132+0000 7fac96ffd640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84012740 legacy=0x7fac84007b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:26.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:26.132+0000 7faca59e0640 1 -- reap_dead start 2024-04-22T06:34:26.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:26.132+0000 7fac96ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400cb10 0x7fac84007b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:34:26.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:26.132+0000 7fac96ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac8400d6e0 0x7fac84012ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:34:26.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:26.132+0000 7fac96ffd640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca015f440 con 0x7fac84008240 2024-04-22T06:34:26.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:26.132+0000 7fac96ffd640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7faca015b980 con 0x7fac8400d6e0 2024-04-22T06:34:26.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:26.132+0000 7fac96ffd640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fac84008ca0 con 0x7fac8400cb10 2024-04-22T06:34:26.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:26.132+0000 7faca59e0640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac8400d6e0 msgr2=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:26.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:26.132+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400cb10 msgr2=0x7fac84007b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:26.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:26.132+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84008240 legacy=0x7fac84008fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:34:26.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:26.132+0000 7faca59e0640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac8400d6e0 0x7fac84012ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:34:26.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:26.132+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400cb10 0x7fac84007b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:34:26.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:26.333+0000 7faca59e0640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac8400d6e0 msgr2=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:26.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:26.333+0000 7faca59e0640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac8400d6e0 0x7fac84012ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:34:26.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:26.333+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400cb10 msgr2=0x7fac84007b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:26.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:26.333+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84008240 legacy=0x7fac84008fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:34:26.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:26.333+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400cb10 0x7fac84007b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:34:26.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:26.734+0000 7faca59e0640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac8400d6e0 msgr2=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:26.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:26.734+0000 7faca59e0640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac8400d6e0 0x7fac84012ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:34:26.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:26.734+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400cb10 msgr2=0x7fac84007b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:26.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:26.734+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84008240 legacy=0x7fac84008fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:34:26.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:26.734+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400cb10 0x7fac84007b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:34:27.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:27.535+0000 7faca59e0640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac8400d6e0 msgr2=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:27.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:27.535+0000 7faca59e0640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac8400d6e0 0x7fac84012ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:34:27.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:27.535+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84008240 legacy=0x7fac84008fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:34:27.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:27.535+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400cb10 msgr2=0x7fac84007b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:27.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:27.535+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400cb10 0x7fac84007b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:34:29.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:29.132+0000 7fac96ffd640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400cb10 msgr2=0x7fac84007b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:29.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:29.132+0000 7fac96ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400cb10 0x7fac84007b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:34:29.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:29.132+0000 7fac96ffd640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac8400d6e0 msgr2=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:29.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:29.132+0000 7fac96ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac8400d6e0 0x7fac84012ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:34:29.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:29.132+0000 7fac96ffd640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84008240 legacy=0x7fac84008fc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:29.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:29.132+0000 7fac96ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac84007590 0x7fac84013400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:34:29.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:29.132+0000 7fac96ffd640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca015b070 con 0x7fac84005a90 2024-04-22T06:34:29.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:29.132+0000 7fac96ffd640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7faca015b980 con 0x7fac84007590 2024-04-22T06:34:29.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:29.132+0000 7faca51df640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84005a90 legacy=0x7fac840096d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:34:29.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:29.132+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac84007590 msgr2=0x7fac84013400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:34:29.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:29.132+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac84007590 0x7fac84013400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:34:29.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:29.333+0000 7faca51df640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84005a90 legacy=0x7fac840096d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:34:29.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:29.333+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac84007590 msgr2=0x7fac84013400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:34:29.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:29.333+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac84007590 0x7fac84013400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:34:29.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:29.734+0000 7faca51df640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84005a90 legacy=0x7fac840096d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:34:29.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:29.734+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac84007590 msgr2=0x7fac84013400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:34:29.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:29.734+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac84007590 0x7fac84013400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:34:30.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:30.535+0000 7faca51df640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84005a90 legacy=0x7fac840096d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:34:30.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:30.535+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac84007590 msgr2=0x7fac84013400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:34:30.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:30.535+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac84007590 0x7fac84013400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:34:32.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:32.133+0000 7fac96ffd640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac84007590 msgr2=0x7fac84013400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:32.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:32.133+0000 7fac96ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac84007590 0x7fac84013400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:34:32.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:32.133+0000 7fac96ffd640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84005a90 legacy=0x7fac840096d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:32.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:32.133+0000 7faca59e0640 1 -- reap_dead start 2024-04-22T06:34:32.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:32.133+0000 7fac96ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d6e0 0x7fac84006580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:34:32.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:32.133+0000 7fac96ffd640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca015f440 con 0x7fac8400cb10 2024-04-22T06:34:32.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:32.133+0000 7fac96ffd640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca015b070 con 0x7fac84008240 2024-04-22T06:34:32.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:32.133+0000 7fac96ffd640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7faca015b980 con 0x7fac8400d6e0 2024-04-22T06:34:32.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:32.133+0000 7faca49de640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac8400cb10 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:34:32.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:32.133+0000 7faca59e0640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d6e0 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:34:32.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:32.133+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84008240 legacy=0x7fac84008fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:34:32.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:32.133+0000 7faca59e0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d6e0 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:34:32.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:32.334+0000 7faca49de640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac8400cb10 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:34:32.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:32.334+0000 7faca59e0640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d6e0 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:34:32.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:32.334+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84008240 legacy=0x7fac84008fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:34:32.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:32.334+0000 7faca59e0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d6e0 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:34:32.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:32.735+0000 7faca49de640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac8400cb10 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:34:32.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:32.735+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84008240 legacy=0x7fac84008fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:34:32.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:32.735+0000 7faca59e0640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d6e0 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:34:32.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:32.735+0000 7faca59e0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d6e0 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:34:33.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:33.536+0000 7faca49de640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac8400cb10 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:34:33.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:33.536+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84008240 legacy=0x7fac84008fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:34:33.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:33.536+0000 7faca59e0640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d6e0 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:34:33.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:33.536+0000 7faca59e0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d6e0 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:34:35.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:35.133+0000 7fac96ffd640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d6e0 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:35.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:35.133+0000 7fac96ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d6e0 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:34:35.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:35.133+0000 7fac96ffd640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84008240 legacy=0x7fac84008fc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:35.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:35.133+0000 7fac96ffd640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac8400cb10 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:35.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:35.133+0000 7fac96ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84005a90 0x7fac84006ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:34:35.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:35.133+0000 7fac96ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84013700 0x7fac84007000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:34:35.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:35.133+0000 7fac96ffd640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7faca015b980 con 0x7fac84005a90 2024-04-22T06:34:35.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:35.133+0000 7fac96ffd640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fac84008ca0 con 0x7fac84013700 2024-04-22T06:34:35.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:35.133+0000 7faca51df640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84005a90 msgr2=0x7fac84006ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:35.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:35.133+0000 7faca51df640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84005a90 0x7fac84006ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:34:35.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:35.133+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84013700 msgr2=0x7fac84007000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:35.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:35.133+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84013700 0x7fac84007000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:34:35.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:35.334+0000 7faca51df640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84005a90 msgr2=0x7fac84006ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:35.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:35.334+0000 7faca51df640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84005a90 0x7fac84006ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:34:35.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:35.334+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84013700 msgr2=0x7fac84007000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:35.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:35.334+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84013700 0x7fac84007000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:34:35.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:35.735+0000 7faca51df640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84005a90 msgr2=0x7fac84006ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:35.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:35.735+0000 7faca51df640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84005a90 0x7fac84006ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:34:35.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:35.735+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84013700 msgr2=0x7fac84007000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:35.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:35.735+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84013700 0x7fac84007000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:34:36.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:36.536+0000 7faca51df640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84005a90 msgr2=0x7fac84006ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:36.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:36.536+0000 7faca51df640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84005a90 0x7fac84006ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:34:36.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:36.536+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84013700 msgr2=0x7fac84007000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:36.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:36.536+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84013700 0x7fac84007000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:34:38.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:38.133+0000 7fac96ffd640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84013700 msgr2=0x7fac84007000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:38.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:38.133+0000 7fac96ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84013700 0x7fac84007000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:34:38.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:38.133+0000 7fac96ffd640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84005a90 msgr2=0x7fac84006ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:38.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:38.133+0000 7fac96ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84005a90 0x7fac84006ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:34:38.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:38.133+0000 7fac96ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84007540 0x7fac84007920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:34:38.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:38.133+0000 7faca59e0640 1 -- reap_dead start 2024-04-22T06:34:38.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:38.133+0000 7fac96ffd640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca015f160 con 0x7fac8400cb10 2024-04-22T06:34:38.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:38.133+0000 7fac96ffd640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca015f440 con 0x7fac84008240 2024-04-22T06:34:38.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:38.133+0000 7fac96ffd640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7faca015b980 con 0x7fac84007540 2024-04-22T06:34:38.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:38.133+0000 7faca49de640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84007540 msgr2=0x7fac84007920 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:38.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:38.133+0000 7faca59e0640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac8400cb10 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:34:38.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:38.133+0000 7faca49de640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84007540 0x7fac84007920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:34:38.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:38.133+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84008240 legacy=0x7fac84008fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:34:38.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:38.334+0000 7faca59e0640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac8400cb10 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:34:38.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:38.334+0000 7faca49de640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84007540 msgr2=0x7fac84007920 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:38.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:38.334+0000 7faca49de640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84007540 0x7fac84007920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:34:38.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:38.334+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84008240 legacy=0x7fac84008fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:34:38.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:38.735+0000 7faca49de640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84007540 msgr2=0x7fac84007920 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:38.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:38.735+0000 7faca59e0640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac8400cb10 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:34:38.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:38.735+0000 7faca49de640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84007540 0x7fac84007920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:34:38.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:38.735+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84008240 legacy=0x7fac84008fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:34:39.096 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T06:34:39.100 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T06:34:39.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:39.536+0000 7faca49de640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84007540 msgr2=0x7fac84007920 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:39.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:39.536+0000 7faca59e0640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac8400cb10 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:34:39.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:39.536+0000 7faca49de640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84007540 0x7fac84007920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:34:39.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:39.536+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84008240 legacy=0x7fac84008fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:34:41.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:41.134+0000 7fac96ffd640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84007540 msgr2=0x7fac84007920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:41.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:41.134+0000 7fac96ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84007540 0x7fac84007920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:34:41.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:41.134+0000 7fac96ffd640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84008240 legacy=0x7fac84008fc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:41.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:41.134+0000 7fac96ffd640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac8400cb10 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:41.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:41.134+0000 7fac96ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84007e60 0x7fac84006580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:34:41.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:41.134+0000 7fac96ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d6e0 0x7fac84006ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:34:41.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:41.134+0000 7fac96ffd640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7faca015b980 con 0x7fac8400d6e0 2024-04-22T06:34:41.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:41.134+0000 7fac96ffd640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fac84008ca0 con 0x7fac84007e60 2024-04-22T06:34:41.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:41.134+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d6e0 msgr2=0x7fac84006ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:34:41.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:41.134+0000 7faca51df640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84007e60 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:41.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:41.134+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d6e0 0x7fac84006ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:34:41.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:41.134+0000 7faca51df640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84007e60 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:34:41.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:41.335+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d6e0 msgr2=0x7fac84006ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:34:41.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:41.335+0000 7faca51df640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84007e60 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:41.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:41.335+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d6e0 0x7fac84006ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:34:41.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:41.335+0000 7faca51df640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84007e60 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:34:41.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:41.736+0000 7faca51df640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84007e60 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:41.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:41.736+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d6e0 msgr2=0x7fac84006ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:34:41.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:41.736+0000 7faca51df640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84007e60 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:34:41.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:41.736+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d6e0 0x7fac84006ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:34:42.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:42.537+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d6e0 msgr2=0x7fac84006ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:34:42.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:42.537+0000 7faca51df640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84007e60 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:42.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:42.537+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d6e0 0x7fac84006ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:34:42.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:42.537+0000 7faca51df640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84007e60 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:34:44.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:44.134+0000 7fac96ffd640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84007e60 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:44.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:44.134+0000 7fac96ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84007e60 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:34:44.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:44.134+0000 7fac96ffd640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d6e0 msgr2=0x7fac84006ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:44.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:44.134+0000 7fac96ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400d6e0 0x7fac84006ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:34:44.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:44.134+0000 7fac96ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac84007000 0x7fac84019120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:34:44.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:44.134+0000 7faca59e0640 1 -- reap_dead start 2024-04-22T06:34:44.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:44.134+0000 7fac96ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006ac0 0x7fac84006580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:34:44.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:44.134+0000 7fac96ffd640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca015b070 con 0x7fac840074b0 2024-04-22T06:34:44.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:44.134+0000 7fac96ffd640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7faca015b980 con 0x7fac84007000 2024-04-22T06:34:44.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:44.134+0000 7fac96ffd640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fac84008ca0 con 0x7fac84006ac0 2024-04-22T06:34:44.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:44.134+0000 7faca51df640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac84007000 msgr2=0x7fac84019120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:34:44.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:44.134+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006ac0 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:44.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:44.134+0000 7faca51df640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac84007000 0x7fac84019120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:34:44.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:44.134+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006ac0 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:34:44.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:44.134+0000 7faca59e0640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac840074b0 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:34:44.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:44.335+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006ac0 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:44.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:44.335+0000 7faca59e0640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac840074b0 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:34:44.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:44.335+0000 7faca51df640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac84007000 msgr2=0x7fac84019120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:34:44.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:44.335+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006ac0 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:34:44.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:44.335+0000 7faca51df640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac84007000 0x7fac84019120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:34:44.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:44.736+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006ac0 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:44.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:44.736+0000 7faca51df640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac84007000 msgr2=0x7fac84019120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:34:44.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:44.736+0000 7faca59e0640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac840074b0 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:34:44.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:44.736+0000 7faca51df640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac84007000 0x7fac84019120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:34:44.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:44.736+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006ac0 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:34:45.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:45.537+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006ac0 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:45.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:45.537+0000 7faca51df640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac84007000 msgr2=0x7fac84019120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:34:45.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:45.537+0000 7faca59e0640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac840074b0 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:34:45.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:45.537+0000 7faca51df640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac84007000 0x7fac84019120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:34:45.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:45.537+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006ac0 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:34:47.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:47.134+0000 7fac96ffd640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006ac0 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:47.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:47.134+0000 7fac96ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006ac0 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:34:47.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:47.134+0000 7fac96ffd640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac84007000 msgr2=0x7fac84019120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:47.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:47.134+0000 7fac96ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac84007000 0x7fac84019120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:34:47.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:47.134+0000 7fac96ffd640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac840074b0 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:47.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:47.134+0000 7fac96ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac8400cb10 0x7fac84007890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:34:47.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:47.134+0000 7fac96ffd640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca015f160 con 0x7fac84008240 2024-04-22T06:34:47.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:47.134+0000 7fac96ffd640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7faca015b980 con 0x7fac8400cb10 2024-04-22T06:34:47.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:47.134+0000 7faca49de640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac8400cb10 msgr2=0x7fac84007890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:47.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:47.134+0000 7faca51df640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84008240 legacy=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:34:47.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:47.134+0000 7faca49de640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac8400cb10 0x7fac84007890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:34:47.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:47.335+0000 7faca51df640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84008240 legacy=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:34:47.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:47.335+0000 7faca49de640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac8400cb10 msgr2=0x7fac84007890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:47.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:47.335+0000 7faca49de640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac8400cb10 0x7fac84007890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:34:47.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:47.736+0000 7faca49de640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac8400cb10 msgr2=0x7fac84007890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:47.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:47.736+0000 7faca51df640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84008240 legacy=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:34:47.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:47.736+0000 7faca49de640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac8400cb10 0x7fac84007890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:34:48.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:48.537+0000 7faca51df640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84008240 legacy=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:34:48.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:48.537+0000 7faca49de640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac8400cb10 msgr2=0x7fac84007890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:48.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:48.537+0000 7faca49de640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac8400cb10 0x7fac84007890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:34:50.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:50.135+0000 7fac96ffd640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac8400cb10 msgr2=0x7fac84007890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:50.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:50.135+0000 7fac96ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac8400cb10 0x7fac84007890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:34:50.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:50.135+0000 7fac96ffd640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84008240 legacy=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:50.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:50.135+0000 7fac96ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006ac0 0x7fac84006580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:34:50.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:50.135+0000 7faca59e0640 1 -- reap_dead start 2024-04-22T06:34:50.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:50.135+0000 7fac96ffd640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca015b070 con 0x7fac8400e320 2024-04-22T06:34:50.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:50.135+0000 7fac96ffd640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca015f160 con 0x7fac840074b0 2024-04-22T06:34:50.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:50.135+0000 7fac96ffd640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7faca015b980 con 0x7fac84006ac0 2024-04-22T06:34:50.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:50.135+0000 7faca59e0640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac8400e320 legacy=0x7fac8400e700 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:34:50.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:50.135+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006ac0 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:50.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:50.135+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac840074b0 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:34:50.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:50.135+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006ac0 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:34:50.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:50.336+0000 7faca59e0640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac8400e320 legacy=0x7fac8400e700 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:34:50.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:50.336+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006ac0 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:50.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:50.336+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac840074b0 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:34:50.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:50.336+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006ac0 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:34:50.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:50.737+0000 7faca59e0640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac8400e320 legacy=0x7fac8400e700 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:34:50.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:50.737+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac840074b0 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:34:50.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:50.737+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006ac0 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:50.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:50.737+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006ac0 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:34:51.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:51.538+0000 7faca59e0640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac8400e320 legacy=0x7fac8400e700 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:34:51.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:51.538+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac840074b0 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:34:51.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:51.538+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006ac0 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:51.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:51.538+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006ac0 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:34:53.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:53.134+0000 7fac96ffd640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006ac0 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:53.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:53.134+0000 7fac96ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006ac0 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:34:53.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:53.134+0000 7fac96ffd640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac840074b0 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:53.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:53.134+0000 7fac96ffd640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac8400e320 legacy=0x7fac8400e700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:53.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:53.134+0000 7fac96ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84007000 0x7fac84019120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:34:53.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:53.134+0000 7fac96ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 0x7fac84007890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:34:53.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:53.134+0000 7fac96ffd640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7faca015b980 con 0x7fac84007000 2024-04-22T06:34:53.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:53.134+0000 7fac96ffd640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fac84008ca0 con 0x7fac8400cb10 2024-04-22T06:34:53.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:53.134+0000 7faca51df640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84007000 msgr2=0x7fac84019120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:53.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:53.134+0000 7faca51df640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84007000 0x7fac84019120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:34:53.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:53.134+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 msgr2=0x7fac84007890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:34:53.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:53.134+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 0x7fac84007890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:34:53.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:53.335+0000 7faca51df640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84007000 msgr2=0x7fac84019120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:53.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:53.335+0000 7faca51df640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84007000 0x7fac84019120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:34:53.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:53.335+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 msgr2=0x7fac84007890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:34:53.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:53.335+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 0x7fac84007890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:34:53.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:53.736+0000 7faca51df640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84007000 msgr2=0x7fac84019120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:53.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:53.736+0000 7faca51df640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84007000 0x7fac84019120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:34:53.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:53.736+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 msgr2=0x7fac84007890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:34:53.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:53.736+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 0x7fac84007890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:34:54.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:54.537+0000 7faca51df640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84007000 msgr2=0x7fac84019120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:54.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:54.537+0000 7faca51df640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84007000 0x7fac84019120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:34:54.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:54.537+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 msgr2=0x7fac84007890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:34:54.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:54.537+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 0x7fac84007890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:34:56.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:56.134+0000 7fac96ffd640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 msgr2=0x7fac84007890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:56.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:56.134+0000 7fac96ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 0x7fac84007890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:34:56.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:56.134+0000 7fac96ffd640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84007000 msgr2=0x7fac84019120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:56.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:56.134+0000 7fac96ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84007000 0x7fac84019120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:34:56.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:56.134+0000 7fac96ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006ac0 0x7fac84006580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:34:56.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:56.134+0000 7faca59e0640 1 -- reap_dead start 2024-04-22T06:34:56.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:56.134+0000 7fac96ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84007000 0x7fac84012ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:34:56.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:56.134+0000 7fac96ffd640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca015f440 con 0x7fac8400e320 2024-04-22T06:34:56.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:56.134+0000 7fac96ffd640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7faca015b980 con 0x7fac84007000 2024-04-22T06:34:56.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:56.134+0000 7fac96ffd640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fac84008ca0 con 0x7fac84006ac0 2024-04-22T06:34:56.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:56.134+0000 7faca49de640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac8400e320 legacy=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:34:56.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:56.134+0000 7faca59e0640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84007000 msgr2=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:56.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:56.134+0000 7faca59e0640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84007000 0x7fac84012ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:34:56.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:56.134+0000 7faca51df640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006ac0 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:56.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:56.134+0000 7faca51df640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006ac0 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:34:56.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:56.335+0000 7faca59e0640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84007000 msgr2=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:56.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:56.335+0000 7faca49de640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac8400e320 legacy=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:34:56.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:56.335+0000 7faca59e0640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84007000 0x7fac84012ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:34:56.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:56.335+0000 7faca51df640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006ac0 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:56.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:56.335+0000 7faca51df640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006ac0 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:34:56.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:56.736+0000 7faca59e0640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84007000 msgr2=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:56.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:56.736+0000 7faca49de640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac8400e320 legacy=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:34:56.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:56.736+0000 7faca59e0640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84007000 0x7fac84012ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:34:56.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:56.736+0000 7faca51df640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006ac0 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:56.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:56.736+0000 7faca51df640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006ac0 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:34:57.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:57.537+0000 7faca59e0640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84007000 msgr2=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:34:57.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:57.537+0000 7faca49de640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac8400e320 legacy=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:34:57.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:57.537+0000 7faca59e0640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84007000 0x7fac84012ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:34:57.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:57.537+0000 7faca51df640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006ac0 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:34:57.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:57.537+0000 7faca51df640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006ac0 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:34:59.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:59.134+0000 7fac96ffd640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006ac0 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:59.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:59.134+0000 7fac96ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84006ac0 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:34:59.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:59.134+0000 7fac96ffd640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84007000 msgr2=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:59.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:59.134+0000 7fac96ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84007000 0x7fac84012ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:34:59.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:59.134+0000 7fac96ffd640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac8400e320 legacy=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:34:59.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:59.134+0000 7fac96ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac84010710 0x7fac84010b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:34:59.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:59.134+0000 7fac96ffd640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca015b070 con 0x7fac8400cb10 2024-04-22T06:34:59.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:59.134+0000 7fac96ffd640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7faca015b980 con 0x7fac84010710 2024-04-22T06:34:59.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:59.134+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac8400cb10 legacy=0x7fac840074b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:34:59.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:59.134+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac84010710 msgr2=0x7fac84010b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:34:59.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:59.134+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac84010710 0x7fac84010b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:34:59.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:59.335+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac84010710 msgr2=0x7fac84010b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:34:59.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:59.335+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac8400cb10 legacy=0x7fac840074b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:34:59.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:59.335+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac84010710 0x7fac84010b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:34:59.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:59.736+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac84010710 msgr2=0x7fac84010b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:34:59.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:59.736+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac8400cb10 legacy=0x7fac840074b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:34:59.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:34:59.736+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac84010710 0x7fac84010b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:35:00.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:00.537+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac84010710 msgr2=0x7fac84010b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:00.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:00.537+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac8400cb10 legacy=0x7fac840074b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:00.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:00.537+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac84010710 0x7fac84010b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:35:02.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:02.135+0000 7fac96ffd640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac84010710 msgr2=0x7fac84010b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:02.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:02.135+0000 7fac96ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac84010710 0x7fac84010b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:35:02.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:02.135+0000 7fac96ffd640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac8400cb10 legacy=0x7fac840074b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:02.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:02.135+0000 7fac96ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84007000 0x7fac84012ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:35:02.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:02.135+0000 7faca59e0640 1 -- reap_dead start 2024-04-22T06:35:02.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:02.135+0000 7fac96ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 0x7fac84006580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:35:02.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:02.135+0000 7fac96ffd640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca015f440 con 0x7fac8400e320 2024-04-22T06:35:02.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:02.135+0000 7fac96ffd640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7faca015b980 con 0x7fac84007000 2024-04-22T06:35:02.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:02.135+0000 7fac96ffd640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fac84008ca0 con 0x7fac8400cb10 2024-04-22T06:35:02.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:02.135+0000 7faca49de640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac8400e320 legacy=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:35:02.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:02.135+0000 7faca51df640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84007000 msgr2=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:35:02.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:02.135+0000 7faca51df640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84007000 0x7fac84012ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:35:02.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:02.135+0000 7faca59e0640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:02.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:02.135+0000 7faca59e0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:35:02.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:02.336+0000 7faca51df640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84007000 msgr2=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:35:02.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:02.336+0000 7faca49de640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac8400e320 legacy=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:35:02.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:02.336+0000 7faca51df640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84007000 0x7fac84012ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:35:02.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:02.336+0000 7faca59e0640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:02.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:02.336+0000 7faca59e0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:35:02.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:02.737+0000 7faca51df640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84007000 msgr2=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:35:02.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:02.737+0000 7faca49de640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac8400e320 legacy=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:35:02.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:02.737+0000 7faca51df640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84007000 0x7fac84012ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:35:02.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:02.737+0000 7faca59e0640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:02.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:02.737+0000 7faca59e0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:35:03.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:03.538+0000 7faca49de640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac8400e320 legacy=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:35:03.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:03.538+0000 7faca51df640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84007000 msgr2=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:35:03.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:03.538+0000 7faca51df640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84007000 0x7fac84012ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:35:03.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:03.538+0000 7faca59e0640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:03.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:03.538+0000 7faca59e0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:35:05.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:05.135+0000 7fac96ffd640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:05.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:05.135+0000 7fac96ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:35:05.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:05.135+0000 7fac96ffd640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84007000 msgr2=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:05.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:05.135+0000 7fac96ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84007000 0x7fac84012ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:35:05.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:05.135+0000 7fac96ffd640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac8400e320 legacy=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:05.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:05.135+0000 7fac96ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84010710 0x7fac84010af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:35:05.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:05.135+0000 7fac96ffd640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca015b070 con 0x7fac84006ac0 2024-04-22T06:35:05.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:05.135+0000 7fac96ffd640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7faca015b980 con 0x7fac84010710 2024-04-22T06:35:05.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:05.136+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84010710 msgr2=0x7fac84010af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:35:05.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:05.136+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84006ac0 legacy=0x7fac840074b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:05.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:05.136+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84010710 0x7fac84010af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:35:05.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:05.337+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84006ac0 legacy=0x7fac840074b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:05.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:05.337+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84010710 msgr2=0x7fac84010af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:35:05.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:05.337+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84010710 0x7fac84010af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:35:05.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:05.738+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84006ac0 legacy=0x7fac840074b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:05.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:05.738+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84010710 msgr2=0x7fac84010af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:35:05.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:05.738+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84010710 0x7fac84010af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:35:06.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:06.539+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84010710 msgr2=0x7fac84010af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:35:06.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:06.539+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84006ac0 legacy=0x7fac840074b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:06.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:06.539+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84010710 0x7fac84010af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:35:08.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:08.136+0000 7fac96ffd640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84010710 msgr2=0x7fac84010af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:08.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:08.136+0000 7fac96ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84010710 0x7fac84010af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:35:08.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:08.136+0000 7fac96ffd640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84006ac0 legacy=0x7fac840074b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:08.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:08.136+0000 7fac96ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 0x7fac84006580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:35:08.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:08.136+0000 7faca59e0640 1 -- reap_dead start 2024-04-22T06:35:08.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:08.136+0000 7fac96ffd640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca015f440 con 0x7fac84006ac0 2024-04-22T06:35:08.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:08.136+0000 7fac96ffd640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca015b070 con 0x7fac84007000 2024-04-22T06:35:08.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:08.136+0000 7fac96ffd640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7faca015b980 con 0x7fac8400cb10 2024-04-22T06:35:08.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:08.136+0000 7faca49de640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84006ac0 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:35:08.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:08.136+0000 7faca59e0640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84007000 legacy=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:08.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:08.136+0000 7faca51df640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:08.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:08.136+0000 7faca51df640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:35:08.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:08.337+0000 7faca49de640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84006ac0 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:35:08.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:08.337+0000 7faca51df640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:08.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:08.337+0000 7faca59e0640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84007000 legacy=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:08.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:08.337+0000 7faca51df640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:35:08.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:08.738+0000 7faca49de640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84006ac0 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:35:08.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:08.738+0000 7faca59e0640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84007000 legacy=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:08.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:08.738+0000 7faca51df640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:08.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:08.738+0000 7faca51df640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:35:09.166 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T06:35:09.173 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T06:35:09.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:09.539+0000 7faca49de640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84006ac0 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:35:09.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:09.539+0000 7faca51df640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:09.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:09.539+0000 7faca59e0640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84007000 legacy=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:09.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:09.539+0000 7faca51df640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:35:11.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:11.136+0000 7fac96ffd640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:11.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:11.136+0000 7fac96ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:35:11.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:11.136+0000 7fac96ffd640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84007000 legacy=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:11.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:11.136+0000 7fac96ffd640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84006ac0 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:11.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:11.136+0000 7fac96ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac840074b0 0x7fac840078c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:35:11.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:11.136+0000 7fac96ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 0x7fac84011750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:35:11.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:11.136+0000 7fac96ffd640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7faca015b980 con 0x7fac840074b0 2024-04-22T06:35:11.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:11.136+0000 7fac96ffd640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fac84008ca0 con 0x7fac84011340 2024-04-22T06:35:11.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:11.136+0000 7faca51df640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac840074b0 msgr2=0x7fac840078c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:35:11.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:11.136+0000 7faca51df640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac840074b0 0x7fac840078c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:35:11.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:11.136+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 msgr2=0x7fac84011750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:35:11.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:11.136+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 0x7fac84011750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:35:11.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:11.337+0000 7faca51df640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac840074b0 msgr2=0x7fac840078c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:35:11.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:11.337+0000 7faca51df640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac840074b0 0x7fac840078c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:35:11.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:11.337+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 msgr2=0x7fac84011750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:35:11.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:11.337+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 0x7fac84011750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:35:11.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:11.738+0000 7faca51df640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac840074b0 msgr2=0x7fac840078c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:35:11.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:11.738+0000 7faca51df640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac840074b0 0x7fac840078c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:35:11.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:11.738+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 msgr2=0x7fac84011750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:35:11.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:11.738+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 0x7fac84011750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:35:12.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:12.539+0000 7faca51df640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac840074b0 msgr2=0x7fac840078c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:35:12.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:12.539+0000 7faca51df640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac840074b0 0x7fac840078c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:35:12.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:12.539+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 msgr2=0x7fac84011750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:35:12.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:12.539+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 0x7fac84011750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:35:14.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:14.136+0000 7fac96ffd640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 msgr2=0x7fac84011750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:14.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:14.136+0000 7fac96ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 0x7fac84011750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:35:14.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:14.136+0000 7fac96ffd640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac840074b0 msgr2=0x7fac840078c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:14.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:14.136+0000 7fac96ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac840074b0 0x7fac840078c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:35:14.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:14.136+0000 7fac96ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 0x7fac84006580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:35:14.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:14.136+0000 7fac96ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84011c90 0x7fac8401a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:35:14.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:14.136+0000 7fac96ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 0x7fac84011750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:35:14.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:14.136+0000 7fac96ffd640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7faca015b980 con 0x7fac84011c90 2024-04-22T06:35:14.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:14.136+0000 7fac96ffd640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fac84008ca0 con 0x7fac8400cb10 2024-04-22T06:35:14.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:14.136+0000 7faca59e0640 1 -- reap_dead start 2024-04-22T06:35:14.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:14.136+0000 7fac96ffd640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fac84012070 con 0x7fac84011340 2024-04-22T06:35:14.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:14.136+0000 7faca49de640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84011c90 msgr2=0x7fac8401a120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:35:14.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:14.136+0000 7faca49de640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84011c90 0x7fac8401a120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:35:14.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:14.136+0000 7faca51df640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:14.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:14.136+0000 7faca51df640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:35:14.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:14.136+0000 7faca59e0640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 msgr2=0x7fac84011750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:35:14.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:14.136+0000 7faca59e0640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 0x7fac84011750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:35:14.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:14.337+0000 7faca49de640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84011c90 msgr2=0x7fac8401a120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:35:14.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:14.337+0000 7faca49de640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84011c90 0x7fac8401a120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:35:14.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:14.337+0000 7faca51df640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:14.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:14.337+0000 7faca59e0640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 msgr2=0x7fac84011750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:35:14.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:14.337+0000 7faca51df640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:35:14.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:14.337+0000 7faca59e0640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 0x7fac84011750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:35:14.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:14.738+0000 7faca49de640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84011c90 msgr2=0x7fac8401a120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:35:14.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:14.738+0000 7faca49de640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84011c90 0x7fac8401a120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:35:14.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:14.738+0000 7faca59e0640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 msgr2=0x7fac84011750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:35:14.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:14.738+0000 7faca51df640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:14.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:14.738+0000 7faca59e0640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 0x7fac84011750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:35:14.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:14.738+0000 7faca51df640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:35:15.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:15.539+0000 7faca49de640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84011c90 msgr2=0x7fac8401a120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:35:15.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:15.539+0000 7faca49de640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84011c90 0x7fac8401a120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:35:15.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:15.539+0000 7faca59e0640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 msgr2=0x7fac84011750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:35:15.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:15.539+0000 7faca51df640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:15.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:15.539+0000 7faca59e0640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 0x7fac84011750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:35:15.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:15.539+0000 7faca51df640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:35:17.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:17.136+0000 7fac96ffd640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 msgr2=0x7fac84011750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:17.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:17.136+0000 7fac96ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 0x7fac84011750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:35:17.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:17.136+0000 7fac96ffd640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:17.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:17.136+0000 7fac96ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:35:17.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:17.136+0000 7fac96ffd640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84011c90 msgr2=0x7fac8401a120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:17.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:17.136+0000 7fac96ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84011c90 0x7fac8401a120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:35:17.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:17.136+0000 7fac96ffd640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca015f160 con 0x7fac84007000 2024-04-22T06:35:17.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:17.136+0000 7fac96ffd640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca015f440 con 0x7fac84006ac0 2024-04-22T06:35:17.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:17.136+0000 7faca49de640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84007000 legacy=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:35:17.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:17.136+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84006ac0 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:17.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:17.337+0000 7faca49de640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84007000 legacy=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:35:17.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:17.338+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84006ac0 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:17.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:17.738+0000 7faca49de640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84007000 legacy=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:35:17.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:17.739+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84006ac0 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:18.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:18.539+0000 7faca49de640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84007000 legacy=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:35:18.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:18.540+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84006ac0 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:20.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:20.136+0000 7fac96ffd640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84006ac0 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:20.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:20.136+0000 7fac96ffd640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84007000 legacy=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:20.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:20.137+0000 7fac96ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 0x7fac84006580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:35:20.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:20.137+0000 7fac96ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 0x7fac84011750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:35:20.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:20.137+0000 7faca59e0640 1 -- reap_dead start 2024-04-22T06:35:20.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:20.137+0000 7fac96ffd640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca015b070 con 0x7fac84006ac0 2024-04-22T06:35:20.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:20.137+0000 7fac96ffd640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7faca015b980 con 0x7fac8400cb10 2024-04-22T06:35:20.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:20.137+0000 7fac96ffd640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fac84008ca0 con 0x7fac84011340 2024-04-22T06:35:20.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:20.137+0000 7faca59e0640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84006ac0 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:20.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:20.137+0000 7faca51df640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:20.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:20.137+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 msgr2=0x7fac84011750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:35:20.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:20.137+0000 7faca51df640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:35:20.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:20.137+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 0x7fac84011750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:35:20.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:20.338+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 msgr2=0x7fac84011750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:35:20.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:20.338+0000 7faca51df640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:20.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:20.338+0000 7faca59e0640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84006ac0 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:20.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:20.338+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 0x7fac84011750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:35:20.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:20.338+0000 7faca51df640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:35:20.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:20.739+0000 7faca51df640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:20.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:20.739+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 msgr2=0x7fac84011750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:35:20.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:20.739+0000 7faca59e0640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84006ac0 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:20.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:20.739+0000 7faca51df640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:35:20.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:20.739+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 0x7fac84011750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:35:21.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:21.540+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 msgr2=0x7fac84011750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:35:21.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:21.540+0000 7faca51df640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:21.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:21.540+0000 7faca59e0640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84006ac0 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:21.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:21.540+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 0x7fac84011750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:35:21.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:21.540+0000 7faca51df640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:35:23.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:23.137+0000 7fac96ffd640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 msgr2=0x7fac84011750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:23.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:23.137+0000 7fac96ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 0x7fac84011750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:35:23.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:23.137+0000 7fac96ffd640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 msgr2=0x7fac84006580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:23.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:23.137+0000 7fac96ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac8400cb10 0x7fac84006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:35:23.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:23.137+0000 7fac96ffd640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84006ac0 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:23.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:23.137+0000 7fac96ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84006fd0 0x7fac840078c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:35:23.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:23.137+0000 7fac96ffd640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca015f160 con 0x7fac84011c90 2024-04-22T06:35:23.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:23.137+0000 7fac96ffd640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7faca015b980 con 0x7fac84006fd0 2024-04-22T06:35:23.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:23.137+0000 7faca49de640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84006fd0 msgr2=0x7fac840078c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:35:23.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:23.137+0000 7faca51df640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84011c90 legacy=0x7fac84008fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:35:23.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:23.137+0000 7faca49de640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84006fd0 0x7fac840078c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:35:23.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:23.338+0000 7faca49de640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84006fd0 msgr2=0x7fac840078c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:35:23.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:23.338+0000 7faca51df640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84011c90 legacy=0x7fac84008fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:35:23.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:23.338+0000 7faca49de640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84006fd0 0x7fac840078c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:35:23.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:23.739+0000 7faca51df640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84011c90 legacy=0x7fac84008fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:35:23.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:23.739+0000 7faca49de640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84006fd0 msgr2=0x7fac840078c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:35:23.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:23.739+0000 7faca49de640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84006fd0 0x7fac840078c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:35:24.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:24.540+0000 7faca49de640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84006fd0 msgr2=0x7fac840078c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:35:24.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:24.540+0000 7faca51df640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84011c90 legacy=0x7fac84008fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:35:24.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:24.540+0000 7faca49de640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84006fd0 0x7fac840078c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:35:26.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:26.137+0000 7fac96ffd640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84006fd0 msgr2=0x7fac840078c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:26.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:26.137+0000 7fac96ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84006fd0 0x7fac840078c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:35:26.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:26.137+0000 7fac96ffd640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84011c90 legacy=0x7fac84008fc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:26.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:26.137+0000 7fac96ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 0x7fac84011750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:35:26.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:26.137+0000 7faca59e0640 1 -- reap_dead start 2024-04-22T06:35:26.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:26.137+0000 7fac96ffd640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca015b070 con 0x7fac84018110 2024-04-22T06:35:26.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:26.137+0000 7fac96ffd640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca015f160 con 0x7fac84006fd0 2024-04-22T06:35:26.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:26.137+0000 7fac96ffd640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7faca015b980 con 0x7fac84011340 2024-04-22T06:35:26.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:26.137+0000 7faca49de640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84018110 legacy=0x7fac8401a500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:35:26.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:26.137+0000 7faca51df640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 msgr2=0x7fac84011750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:35:26.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:26.137+0000 7faca59e0640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84006fd0 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:26.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:26.137+0000 7faca51df640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 0x7fac84011750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:35:26.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:26.338+0000 7faca49de640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84018110 legacy=0x7fac8401a500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:35:26.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:26.338+0000 7faca51df640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 msgr2=0x7fac84011750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:35:26.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:26.338+0000 7faca59e0640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84006fd0 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:26.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:26.338+0000 7faca51df640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 0x7fac84011750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:35:26.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:26.739+0000 7faca49de640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84018110 legacy=0x7fac8401a500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:35:26.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:26.739+0000 7faca51df640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 msgr2=0x7fac84011750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:35:26.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:26.739+0000 7faca59e0640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84006fd0 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:26.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:26.739+0000 7faca51df640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 0x7fac84011750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:35:27.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:27.540+0000 7faca49de640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84018110 legacy=0x7fac8401a500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:35:27.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:27.540+0000 7faca59e0640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84006fd0 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:27.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:27.540+0000 7faca51df640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 msgr2=0x7fac84011750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:35:27.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:27.540+0000 7faca51df640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 0x7fac84011750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:35:29.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:29.138+0000 7fac96ffd640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 msgr2=0x7fac84011750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:29.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:29.138+0000 7fac96ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84011340 0x7fac84011750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:35:29.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:29.138+0000 7fac96ffd640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84006fd0 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:29.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:29.138+0000 7fac96ffd640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84018110 legacy=0x7fac8401a500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:29.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:29.138+0000 7fac96ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84007890 0x7fac84009fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:35:29.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:29.138+0000 7fac96ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac84006580 0x7fac8400e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:35:29.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:29.138+0000 7fac96ffd640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7faca015b980 con 0x7fac84007890 2024-04-22T06:35:29.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:29.138+0000 7fac96ffd640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fac84008ca0 con 0x7fac84006580 2024-04-22T06:35:29.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:29.138+0000 7faca51df640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84007890 msgr2=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:35:29.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:29.138+0000 7faca51df640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84007890 0x7fac84009fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:35:29.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:29.138+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac84006580 msgr2=0x7fac8400e320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:29.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:29.138+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac84006580 0x7fac8400e320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:35:29.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:29.339+0000 7faca51df640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84007890 msgr2=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:35:29.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:29.339+0000 7faca51df640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84007890 0x7fac84009fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:35:29.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:29.339+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac84006580 msgr2=0x7fac8400e320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:29.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:29.339+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac84006580 0x7fac8400e320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:35:29.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:29.740+0000 7faca51df640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84007890 msgr2=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:35:29.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:29.740+0000 7faca51df640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84007890 0x7fac84009fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:35:29.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:29.740+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac84006580 msgr2=0x7fac8400e320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:29.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:29.740+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac84006580 0x7fac8400e320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:35:30.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:30.541+0000 7faca51df640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84007890 msgr2=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:35:30.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:30.541+0000 7faca51df640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84007890 0x7fac84009fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:35:30.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:30.541+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac84006580 msgr2=0x7fac8400e320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:30.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:30.541+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac84006580 0x7fac8400e320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:35:32.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:32.138+0000 7fac96ffd640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac84006580 msgr2=0x7fac8400e320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:32.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:32.138+0000 7fac96ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac84006580 0x7fac8400e320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:35:32.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:32.138+0000 7fac96ffd640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84007890 msgr2=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:32.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:32.138+0000 7fac96ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84007890 0x7fac84009fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:35:32.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:32.138+0000 7faca59e0640 1 -- reap_dead start 2024-04-22T06:35:32.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:32.138+0000 7fac96ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac84006fd0 0x7fac84006960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:35:32.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:32.138+0000 7fac96ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84007890 0x7fac84009fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:35:32.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:32.138+0000 7fac96ffd640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca015f440 con 0x7fac84006580 2024-04-22T06:35:32.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:32.138+0000 7fac96ffd640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7faca015b980 con 0x7fac84007890 2024-04-22T06:35:32.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:32.138+0000 7fac96ffd640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fac84008ca0 con 0x7fac84006fd0 2024-04-22T06:35:32.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:32.138+0000 7faca51df640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84006580 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:35:32.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:32.138+0000 7faca59e0640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84007890 msgr2=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:35:32.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:32.138+0000 7faca59e0640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84007890 0x7fac84009fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:35:32.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:32.138+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac84006fd0 msgr2=0x7fac84006960 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:32.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:32.138+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac84006fd0 0x7fac84006960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:35:32.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:32.339+0000 7faca59e0640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84007890 msgr2=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:35:32.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:32.339+0000 7faca51df640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84006580 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:35:32.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:32.339+0000 7faca59e0640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84007890 0x7fac84009fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:35:32.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:32.339+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac84006fd0 msgr2=0x7fac84006960 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:32.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:32.339+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac84006fd0 0x7fac84006960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:35:32.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:32.740+0000 7faca51df640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84006580 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:35:32.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:32.740+0000 7faca59e0640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84007890 msgr2=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:35:32.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:32.740+0000 7faca59e0640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84007890 0x7fac84009fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:35:32.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:32.740+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac84006fd0 msgr2=0x7fac84006960 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:32.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:32.740+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac84006fd0 0x7fac84006960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:35:33.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:33.541+0000 7faca51df640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84006580 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:35:33.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:33.541+0000 7faca59e0640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84007890 msgr2=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:35:33.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:33.541+0000 7faca59e0640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84007890 0x7fac84009fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:35:33.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:33.541+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac84006fd0 msgr2=0x7fac84006960 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:33.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:33.541+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac84006fd0 0x7fac84006960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:35:35.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:35.138+0000 7fac96ffd640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac84006fd0 msgr2=0x7fac84006960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:35.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:35.138+0000 7fac96ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac84006fd0 0x7fac84006960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:35:35.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:35.138+0000 7fac96ffd640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac84007890 msgr2=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:35.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:35.138+0000 7fac96ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac84007890 0x7fac84009fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:35:35.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:35.138+0000 7fac96ffd640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84006580 legacy=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:35.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:35.138+0000 7fac96ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400cb10 0x7fac84010330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:35:35.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:35.138+0000 7fac96ffd640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca015b070 con 0x7fac84010870 2024-04-22T06:35:35.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:35.138+0000 7fac96ffd640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7faca015b980 con 0x7fac8400cb10 2024-04-22T06:35:35.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:35.138+0000 7faca51df640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400cb10 msgr2=0x7fac84010330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:35:35.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:35.138+0000 7faca49de640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84010870 legacy=0x7fac84010c80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:35.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:35.138+0000 7faca51df640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400cb10 0x7fac84010330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:35:35.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:35.339+0000 7faca49de640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84010870 legacy=0x7fac84010c80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:35.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:35.339+0000 7faca51df640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400cb10 msgr2=0x7fac84010330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:35:35.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:35.339+0000 7faca51df640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400cb10 0x7fac84010330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:35:35.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:35.740+0000 7faca51df640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400cb10 msgr2=0x7fac84010330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:35:35.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:35.740+0000 7faca49de640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84010870 legacy=0x7fac84010c80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:35.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:35.740+0000 7faca51df640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400cb10 0x7fac84010330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:35:36.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:36.541+0000 7faca51df640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400cb10 msgr2=0x7fac84010330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:35:36.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:36.541+0000 7faca49de640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84010870 legacy=0x7fac84010c80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:36.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:36.541+0000 7faca51df640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400cb10 0x7fac84010330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:35:38.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:38.139+0000 7fac96ffd640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400cb10 msgr2=0x7fac84010330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:38.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:38.139+0000 7fac96ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac8400cb10 0x7fac84010330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:35:38.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:38.139+0000 7fac96ffd640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84010870 legacy=0x7fac84010c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:38.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:38.139+0000 7faca59e0640 1 -- reap_dead start 2024-04-22T06:35:38.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:38.139+0000 7fac96ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84007890 0x7fac84012ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:35:38.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:38.139+0000 7fac96ffd640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca015f440 con 0x7fac84006fd0 2024-04-22T06:35:38.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:38.139+0000 7fac96ffd640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca015b070 con 0x7fac84011390 2024-04-22T06:35:38.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:38.139+0000 7fac96ffd640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7faca015b980 con 0x7fac84007890 2024-04-22T06:35:38.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:38.139+0000 7faca59e0640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84006fd0 legacy=0x7fac84010c50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:35:38.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:38.139+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84007890 msgr2=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:35:38.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:38.139+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84011390 legacy=0x7fac8401b130 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:38.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:38.139+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84007890 0x7fac84012ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:35:38.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:38.340+0000 7faca59e0640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84006fd0 legacy=0x7fac84010c50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:35:38.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:38.340+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84011390 legacy=0x7fac8401b130 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:38.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:38.340+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84007890 msgr2=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:35:38.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:38.340+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84007890 0x7fac84012ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:35:38.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:38.741+0000 7faca59e0640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84006fd0 legacy=0x7fac84010c50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:35:38.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:38.741+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84011390 legacy=0x7fac8401b130 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:38.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:38.741+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84007890 msgr2=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:35:38.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:38.741+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84007890 0x7fac84012ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:35:39.310 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T06:35:39.313 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T06:35:39.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:39.542+0000 7faca59e0640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84006fd0 legacy=0x7fac84010c50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:35:39.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:39.542+0000 7faca49de640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84007890 msgr2=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:35:39.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:39.542+0000 7faca51df640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84011390 legacy=0x7fac8401b130 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:39.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:39.542+0000 7faca49de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84007890 0x7fac84012ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:35:41.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:41.139+0000 7fac96ffd640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84007890 msgr2=0x7fac84012ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:41.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:41.139+0000 7fac96ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac84007890 0x7fac84012ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:35:41.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:41.139+0000 7fac96ffd640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fac84011390 legacy=0x7fac8401b130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:41.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:41.139+0000 7fac96ffd640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fac84006fd0 legacy=0x7fac84010c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:41.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:41.139+0000 7fac96ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac8400cb10 0x7fac84010330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:35:41.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:41.139+0000 7fac96ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac84011770 0x7fac84009fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:35:41.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:41.139+0000 7fac96ffd640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7faca015b980 con 0x7fac8400cb10 2024-04-22T06:35:41.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:41.139+0000 7fac96ffd640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fac84008ca0 con 0x7fac84011770 2024-04-22T06:35:41.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:41.139+0000 7faca51df640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac8400cb10 msgr2=0x7fac84010330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:35:41.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:41.139+0000 7faca51df640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac8400cb10 0x7fac84010330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:35:41.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:41.139+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac84011770 msgr2=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:41.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:41.139+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac84011770 0x7fac84009fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:35:41.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:41.340+0000 7faca51df640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac8400cb10 msgr2=0x7fac84010330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:35:41.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:41.340+0000 7faca51df640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac8400cb10 0x7fac84010330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:35:41.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:41.340+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac84011770 msgr2=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:41.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:41.340+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac84011770 0x7fac84009fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:35:41.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:41.741+0000 7faca51df640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac8400cb10 msgr2=0x7fac84010330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:35:41.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:41.741+0000 7faca51df640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac8400cb10 0x7fac84010330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:35:41.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:41.741+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac84011770 msgr2=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:41.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:41.741+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac84011770 0x7fac84009fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:35:42.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:42.542+0000 7faca51df640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fac8400cb10 msgr2=0x7fac84010330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:35:42.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:42.542+0000 7faca51df640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fac8400cb10 0x7fac84010330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:35:42.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:42.542+0000 7faca49de640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fac84011770 msgr2=0x7fac84009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:42.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:42.542+0000 7faca49de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac84011770 0x7fac84009fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:35:44.054 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-04-22T06:35:44.055 ERROR:teuthology.run_tasks:Manager failed: quiescer Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 310, in task stop_all_quiescers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 260, in stop_all_quiescers thrasher.stop() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 252, in stop self.tell_quiesce_leader( "--cancel", "--all" ) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 110, in tell_quiesce_leader leader = self.fs.get_var('qdb_leader') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 959, in get_var return self.get_mds_map(status=status)[var] File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 955, in get_mds_map status = self.status() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 408, in status return FSStatus(self.mon_manager, epoch) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 88, in __init__ self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout return self.run_ceph_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd return self.mon_manager.run_cluster_cmd(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' 2024-04-22T06:35:44.057 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-22T06:35:44.068 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-22T06:35:44.068 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T06:35:44.068 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-22T06:35:44.088 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056883,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19611 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-04-22T06:35:44.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T06:35:44.088 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-04-22T06:35:44.146 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-22T06:35:44.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T06:35:44.146 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-22T06:35:44.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T06:35:44.215 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link delete ceph-brx 2024-04-22T06:35:44.313 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-22T06:35:44.323 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-22T06:35:44.323 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T06:35:44.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:44.430+0000 7f3f18f2e640 1 Processor -- start 2024-04-22T06:35:44.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:44.430+0000 7f3f18f2e640 1 -- start start 2024-04-22T06:35:44.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:44.431+0000 7f3f18f2e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3f140c9970 0x7f3f14071300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:35:44.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:44.431+0000 7f3f18f2e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3f140c62c0 0x7f3f14071840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:35:44.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:44.431+0000 7f3f18f2e640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f140c9e40 con 0x7f3f140c5ee0 2024-04-22T06:35:44.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:44.431+0000 7f3f18f2e640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415d340 con 0x7f3f140c62c0 2024-04-22T06:35:44.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:44.431+0000 7f3f18f2e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415c450 con 0x7f3f140c9970 2024-04-22T06:35:44.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:44.431+0000 7f3f12d9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3f140c62c0 msgr2=0x7f3f14071840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:35:44.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:44.431+0000 7f3f1259c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3f140c5ee0 legacy=0x7f3f141598e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:35:44.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:44.431+0000 7f3f12d9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3f140c62c0 0x7f3f14071840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:35:44.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:44.431+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3f140c9970 msgr2=0x7f3f14071300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:44.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:44.431+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3f140c9970 0x7f3f14071300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:35:44.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:44.632+0000 7f3f12d9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3f140c62c0 msgr2=0x7f3f14071840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:35:44.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:44.632+0000 7f3f1259c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3f140c5ee0 legacy=0x7f3f141598e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:35:44.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:44.632+0000 7f3f12d9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3f140c62c0 0x7f3f14071840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:35:44.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:44.632+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3f140c9970 msgr2=0x7f3f14071300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:44.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:44.632+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3f140c9970 0x7f3f14071300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:35:45.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:45.033+0000 7f3f12d9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3f140c62c0 msgr2=0x7f3f14071840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:35:45.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:45.033+0000 7f3f1259c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3f140c5ee0 legacy=0x7f3f141598e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:35:45.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:45.033+0000 7f3f12d9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3f140c62c0 0x7f3f14071840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:35:45.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:45.033+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3f140c9970 msgr2=0x7f3f14071300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:45.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:45.033+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3f140c9970 0x7f3f14071300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:35:45.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:45.834+0000 7f3f12d9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3f140c62c0 msgr2=0x7f3f14071840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:35:45.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:45.834+0000 7f3f1259c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3f140c5ee0 legacy=0x7f3f141598e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:35:45.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:45.834+0000 7f3f12d9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3f140c62c0 0x7f3f14071840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:35:45.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:45.834+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3f140c9970 msgr2=0x7f3f14071300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:45.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:45.834+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3f140c9970 0x7f3f14071300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:35:47.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:47.432+0000 7f3f03fff640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3f140c9970 msgr2=0x7f3f14071300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:47.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:47.432+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3f140c9970 0x7f3f14071300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:35:47.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:47.432+0000 7f3f03fff640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3f140c62c0 msgr2=0x7f3f14071840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:47.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:47.432+0000 7f3f03fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3f140c62c0 0x7f3f14071840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:35:47.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:47.432+0000 7f3f03fff640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3f140c5ee0 legacy=0x7f3f141598e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:47.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:47.432+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 0x7f3ef4005ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:35:47.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:47.432+0000 7f3f03fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f1415c730 con 0x7f3ef40061b0 2024-04-22T06:35:47.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:47.432+0000 7f3f03fff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3f1415d340 con 0x7f3ef4003690 2024-04-22T06:35:47.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:47.432+0000 7f3f11d9b640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef40061b0 legacy=0x7f3ef4008670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:47.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:47.432+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 msgr2=0x7f3ef4005ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:35:47.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:47.432+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 0x7f3ef4005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:35:47.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:47.633+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 msgr2=0x7f3ef4005ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:35:47.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:47.633+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 0x7f3ef4005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:35:47.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:47.633+0000 7f3f11d9b640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef40061b0 legacy=0x7f3ef4008670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:48.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:48.034+0000 7f3f11d9b640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef40061b0 legacy=0x7f3ef4008670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:48.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:48.034+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 msgr2=0x7f3ef4005ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:35:48.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:48.034+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 0x7f3ef4005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:35:48.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:48.835+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 msgr2=0x7f3ef4005ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:35:48.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:48.835+0000 7f3f11d9b640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef40061b0 legacy=0x7f3ef4008670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:48.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:48.835+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 0x7f3ef4005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:35:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:50.432+0000 7f3f03fff640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 msgr2=0x7f3ef4005ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:50.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:50.432+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 0x7f3ef4005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:35:50.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:50.432+0000 7f3f03fff640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef40061b0 legacy=0x7f3ef4008670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:50.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:50.432+0000 7f3f12d9d640 1 -- reap_dead start 2024-04-22T06:35:50.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:50.432+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 0x7f3ef4005ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:35:50.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:50.432+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4009f40 0x7f3ef4008670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:35:50.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:50.432+0000 7f3f03fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f140c9e40 con 0x7f3ef400a340 2024-04-22T06:35:50.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:50.432+0000 7f3f03fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415d340 con 0x7f3ef4009f40 2024-04-22T06:35:50.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:50.432+0000 7f3f03fff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3f1415c450 con 0x7f3ef4003690 2024-04-22T06:35:50.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:50.432+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 msgr2=0x7f3ef4005ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:35:50.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:50.432+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 0x7f3ef4005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:35:50.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:50.432+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4009f40 msgr2=0x7f3ef4008670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:50.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:50.432+0000 7f3f12d9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef400a340 legacy=0x7f3ef4008da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:50.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:50.432+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4009f40 0x7f3ef4008670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:35:50.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:50.633+0000 7f3f12d9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef400a340 legacy=0x7f3ef4008da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:50.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:50.633+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4009f40 msgr2=0x7f3ef4008670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:50.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:50.633+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 msgr2=0x7f3ef4005ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:35:50.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:50.633+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4009f40 0x7f3ef4008670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:35:50.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:50.633+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 0x7f3ef4005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:35:51.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:51.034+0000 7f3f12d9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef400a340 legacy=0x7f3ef4008da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:51.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:51.034+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 msgr2=0x7f3ef4005ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:35:51.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:51.034+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4009f40 msgr2=0x7f3ef4008670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:51.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:51.034+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 0x7f3ef4005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:35:51.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:51.034+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4009f40 0x7f3ef4008670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:35:51.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:51.835+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 msgr2=0x7f3ef4005ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:35:51.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:51.835+0000 7f3f12d9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef400a340 legacy=0x7f3ef4008da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:51.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:51.835+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4009f40 msgr2=0x7f3ef4008670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:51.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:51.835+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4009f40 0x7f3ef4008670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:35:51.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:51.835+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 0x7f3ef4005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:35:53.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:53.432+0000 7f3f03fff640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 msgr2=0x7f3ef4005ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:53.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:53.432+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 0x7f3ef4005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:35:53.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:53.432+0000 7f3f03fff640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4009f40 msgr2=0x7f3ef4008670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:53.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:53.432+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4009f40 0x7f3ef4008670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:35:53.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:53.432+0000 7f3f03fff640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef400a340 legacy=0x7f3ef4008da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:53.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:53.432+0000 7f3f03fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef40111c0 0x7f3ef40135b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:35:53.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:53.432+0000 7f3f03fff640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f1415c730 con 0x7f3ef400aa50 2024-04-22T06:35:53.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:53.432+0000 7f3f03fff640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415d340 con 0x7f3ef40111c0 2024-04-22T06:35:53.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:53.432+0000 7f3f1259c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef400aa50 legacy=0x7f3ef40094b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:35:53.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:53.432+0000 7f3f11d9b640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef40111c0 msgr2=0x7f3ef40135b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:35:53.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:53.432+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef40111c0 0x7f3ef40135b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:35:53.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:53.633+0000 7f3f11d9b640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef40111c0 msgr2=0x7f3ef40135b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:35:53.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:53.633+0000 7f3f1259c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef400aa50 legacy=0x7f3ef40094b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:35:53.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:53.633+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef40111c0 0x7f3ef40135b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:35:54.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:54.034+0000 7f3f1259c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef400aa50 legacy=0x7f3ef40094b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:35:54.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:54.034+0000 7f3f11d9b640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef40111c0 msgr2=0x7f3ef40135b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:35:54.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:54.034+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef40111c0 0x7f3ef40135b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:35:54.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:54.835+0000 7f3f1259c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef400aa50 legacy=0x7f3ef40094b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:35:54.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:54.835+0000 7f3f11d9b640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef40111c0 msgr2=0x7f3ef40135b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:35:54.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:54.835+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef40111c0 0x7f3ef40135b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:35:56.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:56.432+0000 7f3f03fff640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef40111c0 msgr2=0x7f3ef40135b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:56.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:56.432+0000 7f3f03fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef40111c0 0x7f3ef40135b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:35:56.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:56.432+0000 7f3f03fff640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef400aa50 legacy=0x7f3ef40094b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:56.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:56.432+0000 7f3f12d9d640 1 -- reap_dead start 2024-04-22T06:35:56.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:56.432+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 0x7f3ef4005ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:35:56.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:56.432+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4009f40 0x7f3ef4008670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:35:56.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:56.432+0000 7f3f03fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f140c9e40 con 0x7f3ef4008e40 2024-04-22T06:35:56.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:56.432+0000 7f3f03fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415d340 con 0x7f3ef4009f40 2024-04-22T06:35:56.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:56.432+0000 7f3f03fff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3f1415c450 con 0x7f3ef4003690 2024-04-22T06:35:56.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:56.433+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 msgr2=0x7f3ef4005ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:35:56.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:56.433+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4009f40 msgr2=0x7f3ef4008670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:56.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:56.433+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 0x7f3ef4005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:35:56.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:56.433+0000 7f3f12d9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4008e40 legacy=0x7f3ef4009270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:56.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:56.433+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4009f40 0x7f3ef4008670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:35:56.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:56.634+0000 7f3f12d9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4008e40 legacy=0x7f3ef4009270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:56.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:56.634+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 msgr2=0x7f3ef4005ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:35:56.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:56.634+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4009f40 msgr2=0x7f3ef4008670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:56.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:56.634+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4009f40 0x7f3ef4008670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:35:56.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:56.634+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 0x7f3ef4005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:35:57.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:57.035+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 msgr2=0x7f3ef4005ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:35:57.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:57.035+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 0x7f3ef4005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:35:57.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:57.035+0000 7f3f12d9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4008e40 legacy=0x7f3ef4009270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:57.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:57.035+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4009f40 msgr2=0x7f3ef4008670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:57.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:57.035+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4009f40 0x7f3ef4008670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:35:57.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:57.836+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4009f40 msgr2=0x7f3ef4008670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:35:57.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:57.836+0000 7f3f12d9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4008e40 legacy=0x7f3ef4009270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:35:57.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:57.836+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 msgr2=0x7f3ef4005ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:35:57.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:57.836+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4009f40 0x7f3ef4008670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:35:57.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:57.836+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 0x7f3ef4005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:35:59.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:59.432+0000 7f3f03fff640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 msgr2=0x7f3ef4005ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:59.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:59.432+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 0x7f3ef4005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:35:59.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:59.432+0000 7f3f03fff640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4009f40 msgr2=0x7f3ef4008670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:59.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:59.432+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4009f40 0x7f3ef4008670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:35:59.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:59.432+0000 7f3f03fff640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4008e40 legacy=0x7f3ef4009270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:35:59.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:59.432+0000 7f3f03fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4010cb0 0x7f3ef4018c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:35:59.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:59.432+0000 7f3f03fff640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f1415c730 con 0x7f3ef40145c0 2024-04-22T06:35:59.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:59.432+0000 7f3f03fff640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415d340 con 0x7f3ef4010cb0 2024-04-22T06:35:59.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:59.432+0000 7f3f1259c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef40145c0 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:35:59.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:59.432+0000 7f3f11d9b640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4010cb0 msgr2=0x7f3ef4018c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:35:59.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:59.432+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4010cb0 0x7f3ef4018c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:35:59.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:59.633+0000 7f3f1259c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef40145c0 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:35:59.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:59.633+0000 7f3f11d9b640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4010cb0 msgr2=0x7f3ef4018c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:35:59.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:35:59.633+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4010cb0 0x7f3ef4018c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:36:00.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:00.034+0000 7f3f1259c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef40145c0 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:00.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:00.034+0000 7f3f11d9b640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4010cb0 msgr2=0x7f3ef4018c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:36:00.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:00.034+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4010cb0 0x7f3ef4018c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:36:00.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:00.835+0000 7f3f11d9b640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4010cb0 msgr2=0x7f3ef4018c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:36:00.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:00.835+0000 7f3f1259c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef40145c0 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:00.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:00.835+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4010cb0 0x7f3ef4018c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:36:02.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:02.433+0000 7f3f03fff640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4010cb0 msgr2=0x7f3ef4018c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:02.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:02.433+0000 7f3f03fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4010cb0 0x7f3ef4018c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:36:02.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:02.433+0000 7f3f03fff640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef40145c0 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:02.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:02.433+0000 7f3f12d9d640 1 -- reap_dead start 2024-04-22T06:36:02.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:02.433+0000 7f3f03fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009f40 0x7f3ef4005ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:36:02.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:02.433+0000 7f3f03fff640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f140c9e40 con 0x7f3ef4003690 2024-04-22T06:36:02.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:02.433+0000 7f3f03fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f1415c730 con 0x7f3ef4008da0 2024-04-22T06:36:02.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:02.433+0000 7f3f03fff640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415d340 con 0x7f3ef4009f40 2024-04-22T06:36:02.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:02.433+0000 7f3f1259c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef4003690 legacy=0x7f3ef4009270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:02.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:02.433+0000 7f3f11d9b640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009f40 msgr2=0x7f3ef4005ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:36:02.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:02.433+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009f40 0x7f3ef4005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:36:02.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:02.433+0000 7f3f12d9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4008da0 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:36:02.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:02.634+0000 7f3f1259c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef4003690 legacy=0x7f3ef4009270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:02.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:02.634+0000 7f3f11d9b640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009f40 msgr2=0x7f3ef4005ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:36:02.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:02.634+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009f40 0x7f3ef4005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:36:02.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:02.634+0000 7f3f12d9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4008da0 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:36:03.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:03.035+0000 7f3f11d9b640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009f40 msgr2=0x7f3ef4005ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:36:03.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:03.035+0000 7f3f1259c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef4003690 legacy=0x7f3ef4009270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:03.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:03.035+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009f40 0x7f3ef4005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:36:03.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:03.035+0000 7f3f12d9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4008da0 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:36:03.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:03.836+0000 7f3f1259c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef4003690 legacy=0x7f3ef4009270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:03.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:03.836+0000 7f3f11d9b640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009f40 msgr2=0x7f3ef4005ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:36:03.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:03.836+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009f40 0x7f3ef4005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:36:03.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:03.836+0000 7f3f12d9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4008da0 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:36:05.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:05.433+0000 7f3f03fff640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009f40 msgr2=0x7f3ef4005ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:05.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:05.433+0000 7f3f03fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009f40 0x7f3ef4005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:36:05.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:05.433+0000 7f3f03fff640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4008da0 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:05.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:05.433+0000 7f3f03fff640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef4003690 legacy=0x7f3ef4009270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:05.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:05.433+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4008650 0x7f3ef4016bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:36:05.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:05.433+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef40170f0 0x7f3ef40194e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:36:05.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:05.433+0000 7f3f03fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415d340 con 0x7f3ef40170f0 2024-04-22T06:36:05.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:05.433+0000 7f3f03fff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3f1415c450 con 0x7f3ef4008650 2024-04-22T06:36:05.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:05.433+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4008650 msgr2=0x7f3ef4016bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:36:05.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:05.433+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef40170f0 msgr2=0x7f3ef40194e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:36:05.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:05.433+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef40170f0 0x7f3ef40194e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:36:05.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:05.433+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4008650 0x7f3ef4016bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:36:05.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:05.634+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef40170f0 msgr2=0x7f3ef40194e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:36:05.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:05.634+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4008650 msgr2=0x7f3ef4016bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:36:05.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:05.634+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef40170f0 0x7f3ef40194e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:36:05.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:05.634+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4008650 0x7f3ef4016bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:36:06.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:06.035+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef40170f0 msgr2=0x7f3ef40194e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:36:06.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:06.035+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4008650 msgr2=0x7f3ef4016bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:36:06.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:06.035+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef40170f0 0x7f3ef40194e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:36:06.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:06.035+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4008650 0x7f3ef4016bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:36:06.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:06.836+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4008650 msgr2=0x7f3ef4016bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:36:06.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:06.836+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef40170f0 msgr2=0x7f3ef40194e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:36:06.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:06.836+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef40170f0 0x7f3ef40194e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:36:06.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:06.836+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4008650 0x7f3ef4016bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:36:08.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:08.433+0000 7f3f03fff640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4008650 msgr2=0x7f3ef4016bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:08.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:08.433+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4008650 0x7f3ef4016bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:36:08.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:08.433+0000 7f3f03fff640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef40170f0 msgr2=0x7f3ef40194e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:08.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:08.433+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef40170f0 0x7f3ef40194e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:36:08.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:08.433+0000 7f3f12d9d640 1 -- reap_dead start 2024-04-22T06:36:08.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:08.433+0000 7f3f03fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4003690 0x7f3ef4009270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:36:08.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:08.433+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4009f40 0x7f3ef4005ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:36:08.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:08.433+0000 7f3f03fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ef4010cb0 con 0x7f3ef4008da0 2024-04-22T06:36:08.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:08.433+0000 7f3f03fff640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415d340 con 0x7f3ef4003690 2024-04-22T06:36:08.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:08.433+0000 7f3f03fff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3f1415c450 con 0x7f3ef4009f40 2024-04-22T06:36:08.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:08.433+0000 7f3f11d9b640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4003690 msgr2=0x7f3ef4009270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:36:08.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:08.433+0000 7f3f1259c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4008da0 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:36:08.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:08.433+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4003690 0x7f3ef4009270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:36:08.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:08.433+0000 7f3f12d9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4009f40 msgr2=0x7f3ef4005ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:36:08.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:08.433+0000 7f3f12d9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4009f40 0x7f3ef4005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:36:08.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:08.634+0000 7f3f11d9b640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4003690 msgr2=0x7f3ef4009270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:36:08.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:08.634+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4003690 0x7f3ef4009270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:36:08.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:08.634+0000 7f3f1259c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4008da0 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:36:08.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:08.634+0000 7f3f12d9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4009f40 msgr2=0x7f3ef4005ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:36:08.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:08.634+0000 7f3f12d9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4009f40 0x7f3ef4005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:36:09.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:09.035+0000 7f3f11d9b640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4003690 msgr2=0x7f3ef4009270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:36:09.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:09.035+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4003690 0x7f3ef4009270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:36:09.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:09.035+0000 7f3f1259c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4008da0 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:36:09.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:09.035+0000 7f3f12d9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4009f40 msgr2=0x7f3ef4005ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:36:09.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:09.035+0000 7f3f12d9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4009f40 0x7f3ef4005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:36:09.395 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T06:36:09.399 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T06:36:09.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:09.836+0000 7f3f11d9b640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4003690 msgr2=0x7f3ef4009270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:36:09.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:09.836+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4003690 0x7f3ef4009270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:36:09.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:09.836+0000 7f3f1259c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4008da0 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:36:09.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:09.836+0000 7f3f12d9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4009f40 msgr2=0x7f3ef4005ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:36:09.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:09.836+0000 7f3f12d9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4009f40 0x7f3ef4005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:36:11.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:11.434+0000 7f3f03fff640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4009f40 msgr2=0x7f3ef4005ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:11.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:11.434+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4009f40 0x7f3ef4005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:36:11.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:11.434+0000 7f3f03fff640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4003690 msgr2=0x7f3ef4009270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:11.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:11.434+0000 7f3f03fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4003690 0x7f3ef4009270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:36:11.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:11.434+0000 7f3f03fff640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4008da0 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:11.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:11.434+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4008650 0x7f3ef400b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:36:11.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:11.434+0000 7f3f03fff640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f140c9e40 con 0x7f3ef400b640 2024-04-22T06:36:11.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:11.434+0000 7f3f03fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415d340 con 0x7f3ef4008650 2024-04-22T06:36:11.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:11.434+0000 7f3f11d9b640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef400b640 legacy=0x7f3ef4018b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:11.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:11.434+0000 7f3f1259c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4008650 msgr2=0x7f3ef400b100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:36:11.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:11.434+0000 7f3f1259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4008650 0x7f3ef400b100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:36:11.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:11.635+0000 7f3f11d9b640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef400b640 legacy=0x7f3ef4018b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:11.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:11.635+0000 7f3f1259c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4008650 msgr2=0x7f3ef400b100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:36:11.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:11.635+0000 7f3f1259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4008650 0x7f3ef400b100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:36:12.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:12.036+0000 7f3f11d9b640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef400b640 legacy=0x7f3ef4018b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:12.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:12.036+0000 7f3f1259c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4008650 msgr2=0x7f3ef400b100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:36:12.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:12.036+0000 7f3f1259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4008650 0x7f3ef400b100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:36:12.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:12.837+0000 7f3f11d9b640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef400b640 legacy=0x7f3ef4018b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:12.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:12.837+0000 7f3f1259c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4008650 msgr2=0x7f3ef400b100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:36:12.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:12.837+0000 7f3f1259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4008650 0x7f3ef400b100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:36:14.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:14.434+0000 7f3f03fff640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4008650 msgr2=0x7f3ef400b100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:14.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:14.434+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4008650 0x7f3ef400b100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:36:14.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:14.434+0000 7f3f03fff640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef400b640 legacy=0x7f3ef4018b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:14.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:14.434+0000 7f3f12d9d640 1 -- reap_dead start 2024-04-22T06:36:14.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:14.434+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef401c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:36:14.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:14.434+0000 7f3f03fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 0x7f3ef4009250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:36:14.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:14.434+0000 7f3f03fff640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ef4010cb0 con 0x7f3ef4003690 2024-04-22T06:36:14.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:14.434+0000 7f3f03fff640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415d340 con 0x7f3ef4008da0 2024-04-22T06:36:14.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:14.434+0000 7f3f03fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415c450 con 0x7f3ef400ba50 2024-04-22T06:36:14.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:14.435+0000 7f3f11d9b640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef4003690 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:14.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:14.435+0000 7f3f12d9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 msgr2=0x7f3ef4009250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:36:14.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:14.435+0000 7f3f1259c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 msgr2=0x7f3ef401c2b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:36:14.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:14.435+0000 7f3f1259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef401c2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:36:14.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:14.435+0000 7f3f12d9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 0x7f3ef4009250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:36:14.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:14.636+0000 7f3f12d9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 msgr2=0x7f3ef4009250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:36:14.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:14.636+0000 7f3f11d9b640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef4003690 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:14.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:14.636+0000 7f3f12d9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 0x7f3ef4009250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:36:14.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:14.636+0000 7f3f1259c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 msgr2=0x7f3ef401c2b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:36:14.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:14.636+0000 7f3f1259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef401c2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:36:15.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:15.037+0000 7f3f12d9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 msgr2=0x7f3ef4009250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:36:15.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:15.037+0000 7f3f11d9b640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef4003690 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:15.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:15.037+0000 7f3f12d9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 0x7f3ef4009250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:36:15.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:15.037+0000 7f3f1259c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 msgr2=0x7f3ef401c2b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:36:15.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:15.037+0000 7f3f1259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef401c2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:36:15.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:15.838+0000 7f3f12d9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 msgr2=0x7f3ef4009250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:36:15.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:15.838+0000 7f3f11d9b640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef4003690 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:15.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:15.838+0000 7f3f12d9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 0x7f3ef4009250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:36:15.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:15.838+0000 7f3f1259c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 msgr2=0x7f3ef401c2b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:36:15.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:15.838+0000 7f3f1259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef401c2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:36:17.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:17.435+0000 7f3f03fff640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 msgr2=0x7f3ef401c2b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:17.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:17.435+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef401c2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:36:17.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:17.435+0000 7f3f03fff640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 msgr2=0x7f3ef4009250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:17.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:17.435+0000 7f3f03fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 0x7f3ef4009250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:36:17.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:17.435+0000 7f3f03fff640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef4003690 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:17.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:17.435+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4009f40 0x7f3ef400b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:36:17.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:17.435+0000 7f3f03fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f140c9e40 con 0x7f3ef400b640 2024-04-22T06:36:17.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:17.435+0000 7f3f03fff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3f1415d340 con 0x7f3ef4009f40 2024-04-22T06:36:17.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:17.435+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4009f40 msgr2=0x7f3ef400b100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:36:17.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:17.435+0000 7f3f11d9b640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef400b640 legacy=0x7f3ef40135b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:36:17.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:17.435+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4009f40 0x7f3ef400b100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:36:17.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:17.636+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4009f40 msgr2=0x7f3ef400b100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:36:17.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:17.636+0000 7f3f11d9b640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef400b640 legacy=0x7f3ef40135b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:36:17.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:17.636+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4009f40 0x7f3ef400b100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:36:18.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:18.037+0000 7f3f11d9b640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef400b640 legacy=0x7f3ef40135b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:36:18.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:18.037+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4009f40 msgr2=0x7f3ef400b100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:36:18.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:18.037+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4009f40 0x7f3ef400b100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:36:18.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:18.838+0000 7f3f11d9b640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef400b640 legacy=0x7f3ef40135b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:36:18.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:18.838+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4009f40 msgr2=0x7f3ef400b100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:36:18.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:18.838+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4009f40 0x7f3ef400b100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:36:20.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:20.435+0000 7f3f03fff640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4009f40 msgr2=0x7f3ef400b100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:20.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:20.435+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4009f40 0x7f3ef400b100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:36:20.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:20.435+0000 7f3f03fff640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef400b640 legacy=0x7f3ef40135b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:20.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:20.435+0000 7f3f03fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 0x7f3ef4009250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:36:20.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:20.435+0000 7f3f12d9d640 1 -- reap_dead start 2024-04-22T06:36:20.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:20.435+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4003690 0x7f3ef400b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:36:20.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:20.435+0000 7f3f03fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ef4010cb0 con 0x7f3ef400ba50 2024-04-22T06:36:20.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:20.435+0000 7f3f03fff640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415d340 con 0x7f3ef4008da0 2024-04-22T06:36:20.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:20.435+0000 7f3f03fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415c450 con 0x7f3ef4003690 2024-04-22T06:36:20.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:20.435+0000 7f3f1259c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 msgr2=0x7f3ef4009250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:36:20.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:20.435+0000 7f3f1259c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 0x7f3ef4009250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:36:20.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:20.435+0000 7f3f12d9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef400ba50 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:36:20.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:20.435+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4003690 msgr2=0x7f3ef400b100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:36:20.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:20.435+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4003690 0x7f3ef400b100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:36:20.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:20.636+0000 7f3f1259c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 msgr2=0x7f3ef4009250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:36:20.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:20.636+0000 7f3f1259c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 0x7f3ef4009250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:36:20.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:20.636+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4003690 msgr2=0x7f3ef400b100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:36:20.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:20.636+0000 7f3f12d9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef400ba50 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:36:20.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:20.636+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4003690 0x7f3ef400b100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:36:21.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:21.037+0000 7f3f1259c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 msgr2=0x7f3ef4009250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:36:21.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:21.037+0000 7f3f1259c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 0x7f3ef4009250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:36:21.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:21.037+0000 7f3f12d9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef400ba50 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:36:21.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:21.037+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4003690 msgr2=0x7f3ef400b100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:36:21.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:21.037+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4003690 0x7f3ef400b100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:36:21.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:21.838+0000 7f3f1259c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 msgr2=0x7f3ef4009250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:36:21.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:21.838+0000 7f3f1259c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 0x7f3ef4009250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:36:21.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:21.838+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4003690 msgr2=0x7f3ef400b100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:36:21.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:21.838+0000 7f3f12d9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef400ba50 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:36:21.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:21.838+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4003690 0x7f3ef400b100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:36:23.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:23.435+0000 7f3f03fff640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4003690 msgr2=0x7f3ef400b100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:23.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:23.435+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4003690 0x7f3ef400b100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:36:23.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:23.435+0000 7f3f03fff640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 msgr2=0x7f3ef4009250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:23.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:23.435+0000 7f3f03fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 0x7f3ef4009250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:36:23.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:23.435+0000 7f3f03fff640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef400ba50 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:23.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:23.435+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4018230 0x7f3ef401a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:36:23.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:23.435+0000 7f3f03fff640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f140c9e40 con 0x7f3ef4005ae0 2024-04-22T06:36:23.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:23.435+0000 7f3f03fff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3f1415d340 con 0x7f3ef4018230 2024-04-22T06:36:23.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:23.435+0000 7f3f1259c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef4005ae0 legacy=0x7f3ef4016b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:23.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:23.436+0000 7f3f11d9b640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4018230 msgr2=0x7f3ef401a620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:36:23.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:23.436+0000 7f3f11d9b640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4018230 0x7f3ef401a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:36:23.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:23.636+0000 7f3f1259c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef4005ae0 legacy=0x7f3ef4016b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:23.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:23.637+0000 7f3f11d9b640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4018230 msgr2=0x7f3ef401a620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:36:23.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:23.637+0000 7f3f11d9b640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4018230 0x7f3ef401a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:36:24.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:24.037+0000 7f3f1259c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef4005ae0 legacy=0x7f3ef4016b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:24.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:24.038+0000 7f3f11d9b640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4018230 msgr2=0x7f3ef401a620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:36:24.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:24.038+0000 7f3f11d9b640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4018230 0x7f3ef401a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:36:24.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:24.838+0000 7f3f1259c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef4005ae0 legacy=0x7f3ef4016b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:24.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:24.839+0000 7f3f11d9b640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4018230 msgr2=0x7f3ef401a620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:36:24.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:24.839+0000 7f3f11d9b640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4018230 0x7f3ef401a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:36:26.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:26.436+0000 7f3f03fff640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4018230 msgr2=0x7f3ef401a620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:26.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:26.436+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4018230 0x7f3ef401a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:36:26.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:26.436+0000 7f3f03fff640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef4005ae0 legacy=0x7f3ef4016b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:26.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:26.436+0000 7f3f12d9d640 1 -- reap_dead start 2024-04-22T06:36:26.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:26.436+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 0x7f3ef400b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:36:26.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:26.436+0000 7f3f03fff640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ef4010cb0 con 0x7f3ef400ba50 2024-04-22T06:36:26.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:26.436+0000 7f3f03fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f140c9e40 con 0x7f3ef4008da0 2024-04-22T06:36:26.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:26.436+0000 7f3f03fff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3f1415d340 con 0x7f3ef4003690 2024-04-22T06:36:26.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:26.436+0000 7f3f12d9d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef400ba50 legacy=0x7f3ef4009180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:26.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:26.436+0000 7f3f11d9b640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4008da0 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:36:26.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:26.436+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 msgr2=0x7f3ef400b100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:36:26.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:26.436+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 0x7f3ef400b100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:36:26.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:26.637+0000 7f3f12d9d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef400ba50 legacy=0x7f3ef4009180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:26.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:26.637+0000 7f3f11d9b640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4008da0 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:36:26.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:26.637+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 msgr2=0x7f3ef400b100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:36:26.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:26.637+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 0x7f3ef400b100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:36:27.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:27.038+0000 7f3f12d9d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef400ba50 legacy=0x7f3ef4009180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:27.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:27.038+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 msgr2=0x7f3ef400b100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:36:27.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:27.038+0000 7f3f11d9b640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4008da0 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:36:27.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:27.038+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 0x7f3ef400b100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:36:27.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:27.839+0000 7f3f12d9d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef400ba50 legacy=0x7f3ef4009180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:27.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:27.839+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 msgr2=0x7f3ef400b100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:36:27.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:27.839+0000 7f3f11d9b640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4008da0 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:36:27.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:27.839+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 0x7f3ef400b100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:36:29.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:29.436+0000 7f3f03fff640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 msgr2=0x7f3ef400b100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:29.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:29.436+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 0x7f3ef400b100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:36:29.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:29.436+0000 7f3f03fff640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4008da0 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:29.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:29.436+0000 7f3f03fff640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef400ba50 legacy=0x7f3ef4009180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:29.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:29.436+0000 7f3f03fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4005ae0 0x7f3ef4017b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:36:29.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:29.436+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4018060 0x7f3ef401a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:36:29.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:29.436+0000 7f3f03fff640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415d340 con 0x7f3ef4005ae0 2024-04-22T06:36:29.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:29.436+0000 7f3f03fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415c450 con 0x7f3ef4018060 2024-04-22T06:36:29.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:29.436+0000 7f3f1259c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4005ae0 msgr2=0x7f3ef4017b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:36:29.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:29.436+0000 7f3f1259c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4005ae0 0x7f3ef4017b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:36:29.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:29.436+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4018060 msgr2=0x7f3ef401a450 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:36:29.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:29.436+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4018060 0x7f3ef401a450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:36:29.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:29.637+0000 7f3f1259c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4005ae0 msgr2=0x7f3ef4017b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:36:29.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:29.637+0000 7f3f1259c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4005ae0 0x7f3ef4017b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:36:29.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:29.637+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4018060 msgr2=0x7f3ef401a450 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:36:29.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:29.637+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4018060 0x7f3ef401a450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:36:30.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:30.038+0000 7f3f1259c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4005ae0 msgr2=0x7f3ef4017b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:36:30.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:30.038+0000 7f3f1259c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4005ae0 0x7f3ef4017b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:36:30.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:30.038+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4018060 msgr2=0x7f3ef401a450 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:36:30.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:30.038+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4018060 0x7f3ef401a450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:36:30.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:30.839+0000 7f3f1259c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4005ae0 msgr2=0x7f3ef4017b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:36:30.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:30.839+0000 7f3f1259c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4005ae0 0x7f3ef4017b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:36:30.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:30.839+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4018060 msgr2=0x7f3ef401a450 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:36:30.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:30.839+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4018060 0x7f3ef401a450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:36:32.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:32.436+0000 7f3f03fff640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4018060 msgr2=0x7f3ef401a450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:32.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:32.436+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4018060 0x7f3ef401a450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:36:32.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:32.436+0000 7f3f03fff640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4005ae0 msgr2=0x7f3ef4017b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:32.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:32.436+0000 7f3f03fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4005ae0 0x7f3ef4017b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:36:32.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:32.436+0000 7f3f12d9d640 1 -- reap_dead start 2024-04-22T06:36:32.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:32.436+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 0x7f3ef400b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:36:32.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:32.436+0000 7f3f03fff640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f1415c730 con 0x7f3ef4003690 2024-04-22T06:36:32.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:32.436+0000 7f3f03fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ef4010cb0 con 0x7f3ef4008da0 2024-04-22T06:36:32.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:32.436+0000 7f3f03fff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3f1415d340 con 0x7f3ef4005ae0 2024-04-22T06:36:32.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:32.436+0000 7f3f11d9b640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef4003690 legacy=0x7f3ef4009180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:32.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:32.436+0000 7f3f1259c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4008da0 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:36:32.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:32.437+0000 7f3f12d9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 msgr2=0x7f3ef400b0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:36:32.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:32.437+0000 7f3f12d9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 0x7f3ef400b0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:36:32.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:32.637+0000 7f3f11d9b640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef4003690 legacy=0x7f3ef4009180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:32.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:32.638+0000 7f3f1259c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4008da0 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:36:32.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:32.638+0000 7f3f12d9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 msgr2=0x7f3ef400b0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:36:32.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:32.638+0000 7f3f12d9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 0x7f3ef400b0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:36:33.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:33.038+0000 7f3f11d9b640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef4003690 legacy=0x7f3ef4009180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:33.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:33.039+0000 7f3f1259c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4008da0 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:36:33.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:33.039+0000 7f3f12d9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 msgr2=0x7f3ef400b0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:36:33.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:33.039+0000 7f3f12d9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 0x7f3ef400b0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:36:33.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:33.839+0000 7f3f11d9b640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef4003690 legacy=0x7f3ef4009180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:33.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:33.840+0000 7f3f1259c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4008da0 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:36:33.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:33.840+0000 7f3f12d9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 msgr2=0x7f3ef400b0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:36:33.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:33.840+0000 7f3f12d9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 0x7f3ef400b0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:36:35.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:35.437+0000 7f3f03fff640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 msgr2=0x7f3ef400b0e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:35.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:35.437+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 0x7f3ef400b0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:36:35.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:35.437+0000 7f3f03fff640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4008da0 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:35.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:35.437+0000 7f3f03fff640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef4003690 legacy=0x7f3ef4009180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:35.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:35.437+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef4007630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:36:35.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:35.437+0000 7f3f03fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007b70 0x7f3ef4007f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:36:35.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:35.437+0000 7f3f03fff640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415d340 con 0x7f3ef4007b70 2024-04-22T06:36:35.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:35.437+0000 7f3f03fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415c450 con 0x7f3ef400ba50 2024-04-22T06:36:35.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:35.437+0000 7f3f11d9b640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007b70 msgr2=0x7f3ef4007f80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:36:35.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:35.437+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007b70 0x7f3ef4007f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:36:35.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:35.437+0000 7f3f1259c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 msgr2=0x7f3ef4007630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:36:35.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:35.437+0000 7f3f1259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef4007630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:36:35.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:35.638+0000 7f3f11d9b640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007b70 msgr2=0x7f3ef4007f80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:36:35.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:35.638+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007b70 0x7f3ef4007f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:36:35.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:35.638+0000 7f3f1259c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 msgr2=0x7f3ef4007630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:36:35.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:35.638+0000 7f3f1259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef4007630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:36:36.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:36.039+0000 7f3f11d9b640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007b70 msgr2=0x7f3ef4007f80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:36:36.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:36.039+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007b70 0x7f3ef4007f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:36:36.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:36.039+0000 7f3f1259c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 msgr2=0x7f3ef4007630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:36:36.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:36.039+0000 7f3f1259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef4007630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:36:36.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:36.840+0000 7f3f11d9b640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007b70 msgr2=0x7f3ef4007f80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:36:36.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:36.840+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007b70 0x7f3ef4007f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:36:36.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:36.840+0000 7f3f1259c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 msgr2=0x7f3ef4007630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:36:36.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:36.840+0000 7f3f1259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef4007630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:36:38.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:38.437+0000 7f3f03fff640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 msgr2=0x7f3ef4007630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:38.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:38.437+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef4007630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:36:38.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:38.437+0000 7f3f03fff640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007b70 msgr2=0x7f3ef4007f80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:38.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:38.437+0000 7f3f03fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007b70 0x7f3ef4007f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:36:38.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:38.437+0000 7f3f12d9d640 1 -- reap_dead start 2024-04-22T06:36:38.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:38.437+0000 7f3f03fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef401a9d0 0x7f3ef401cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:36:38.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:38.437+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4003690 0x7f3ef400b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:36:38.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:38.437+0000 7f3f03fff640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f140c9e40 con 0x7f3ef400ba50 2024-04-22T06:36:38.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:38.437+0000 7f3f03fff640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415d340 con 0x7f3ef401a9d0 2024-04-22T06:36:38.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:38.437+0000 7f3f1259c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef401a9d0 msgr2=0x7f3ef401cdc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:36:38.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:38.437+0000 7f3f03fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415c450 con 0x7f3ef4003690 2024-04-22T06:36:38.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:38.437+0000 7f3f1259c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef401a9d0 0x7f3ef401cdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:36:38.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:38.437+0000 7f3f12d9d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef400ba50 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:38.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:38.437+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4003690 msgr2=0x7f3ef400b0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:36:38.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:38.437+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4003690 0x7f3ef400b0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:36:38.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:38.638+0000 7f3f1259c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef401a9d0 msgr2=0x7f3ef401cdc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:36:38.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:38.638+0000 7f3f12d9d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef400ba50 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:38.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:38.638+0000 7f3f1259c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef401a9d0 0x7f3ef401cdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:36:38.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:38.638+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4003690 msgr2=0x7f3ef400b0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:36:38.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:38.638+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4003690 0x7f3ef400b0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:36:39.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:39.039+0000 7f3f12d9d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef400ba50 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:39.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:39.039+0000 7f3f1259c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef401a9d0 msgr2=0x7f3ef401cdc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:36:39.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:39.039+0000 7f3f1259c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef401a9d0 0x7f3ef401cdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:36:39.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:39.039+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4003690 msgr2=0x7f3ef400b0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:36:39.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:39.039+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4003690 0x7f3ef400b0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:36:39.540 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T06:36:39.543 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T06:36:39.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:39.840+0000 7f3f12d9d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef400ba50 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:39.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:39.840+0000 7f3f1259c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef401a9d0 msgr2=0x7f3ef401cdc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:36:39.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:39.840+0000 7f3f1259c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef401a9d0 0x7f3ef401cdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:36:39.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:39.840+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4003690 msgr2=0x7f3ef400b0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:36:39.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:39.840+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4003690 0x7f3ef400b0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:36:41.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:41.438+0000 7f3f03fff640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4003690 msgr2=0x7f3ef400b0e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:41.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:41.438+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4003690 0x7f3ef400b0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:36:41.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:41.438+0000 7f3f03fff640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef401a9d0 msgr2=0x7f3ef401cdc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:41.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:41.438+0000 7f3f03fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef401a9d0 0x7f3ef401cdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:36:41.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:41.438+0000 7f3f03fff640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef400ba50 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:41.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:41.438+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef40094b0 0x7f3ef4007630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:36:41.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:41.438+0000 7f3f03fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f1415c730 con 0x7f3ef4005ae0 2024-04-22T06:36:41.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:41.438+0000 7f3f03fff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3f1415d340 con 0x7f3ef40094b0 2024-04-22T06:36:41.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:41.438+0000 7f3f11d9b640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef40094b0 msgr2=0x7f3ef4007630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:36:41.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:41.438+0000 7f3f1259c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4005ae0 legacy=0x7f3ef4008da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:36:41.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:41.438+0000 7f3f11d9b640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef40094b0 0x7f3ef4007630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:36:41.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:41.639+0000 7f3f11d9b640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef40094b0 msgr2=0x7f3ef4007630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:36:41.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:41.639+0000 7f3f1259c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4005ae0 legacy=0x7f3ef4008da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:36:41.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:41.639+0000 7f3f11d9b640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef40094b0 0x7f3ef4007630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:36:42.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:42.040+0000 7f3f1259c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4005ae0 legacy=0x7f3ef4008da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:36:42.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:42.040+0000 7f3f11d9b640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef40094b0 msgr2=0x7f3ef4007630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:36:42.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:42.040+0000 7f3f11d9b640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef40094b0 0x7f3ef4007630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:36:42.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:42.841+0000 7f3f1259c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4005ae0 legacy=0x7f3ef4008da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:36:42.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:42.841+0000 7f3f11d9b640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef40094b0 msgr2=0x7f3ef4007630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:36:42.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:42.841+0000 7f3f11d9b640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef40094b0 0x7f3ef4007630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:36:44.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:44.438+0000 7f3f03fff640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef40094b0 msgr2=0x7f3ef4007630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:44.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:44.438+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef40094b0 0x7f3ef4007630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:36:44.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:44.438+0000 7f3f03fff640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4005ae0 legacy=0x7f3ef4008da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:44.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:44.438+0000 7f3f12d9d640 1 -- reap_dead start 2024-04-22T06:36:44.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:44.438+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4003690 0x7f3ef4007f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:36:44.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:44.438+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 0x7f3ef400b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:36:44.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:44.438+0000 7f3f03fff640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f140c9e40 con 0x7f3ef400ba50 2024-04-22T06:36:44.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:44.438+0000 7f3f03fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415d340 con 0x7f3ef4003690 2024-04-22T06:36:44.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:44.438+0000 7f3f03fff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3f1415c450 con 0x7f3ef4005ae0 2024-04-22T06:36:44.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:44.438+0000 7f3f11d9b640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef400ba50 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:44.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:44.438+0000 7f3f12d9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 msgr2=0x7f3ef400b0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:36:44.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:44.438+0000 7f3f1259c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4003690 msgr2=0x7f3ef4007f80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:36:44.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:44.438+0000 7f3f12d9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 0x7f3ef400b0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:36:44.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:44.438+0000 7f3f1259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4003690 0x7f3ef4007f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:36:44.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:44.639+0000 7f3f11d9b640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef400ba50 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:44.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:44.639+0000 7f3f1259c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4003690 msgr2=0x7f3ef4007f80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:36:44.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:44.639+0000 7f3f12d9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 msgr2=0x7f3ef400b0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:36:44.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:44.639+0000 7f3f1259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4003690 0x7f3ef4007f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:36:44.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:44.639+0000 7f3f12d9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 0x7f3ef400b0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:36:45.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:45.040+0000 7f3f11d9b640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef400ba50 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:45.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:45.040+0000 7f3f1259c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4003690 msgr2=0x7f3ef4007f80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:36:45.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:45.040+0000 7f3f12d9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 msgr2=0x7f3ef400b0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:36:45.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:45.040+0000 7f3f1259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4003690 0x7f3ef4007f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:36:45.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:45.040+0000 7f3f12d9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 0x7f3ef400b0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:36:45.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:45.841+0000 7f3f11d9b640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef400ba50 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:45.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:45.841+0000 7f3f12d9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 msgr2=0x7f3ef400b0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:36:45.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:45.841+0000 7f3f1259c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4003690 msgr2=0x7f3ef4007f80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:36:45.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:45.841+0000 7f3f12d9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 0x7f3ef400b0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:36:45.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:45.841+0000 7f3f1259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4003690 0x7f3ef4007f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:36:47.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:47.438+0000 7f3f03fff640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 msgr2=0x7f3ef400b0e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:47.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:47.438+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 0x7f3ef400b0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:36:47.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:47.438+0000 7f3f03fff640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4003690 msgr2=0x7f3ef4007f80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:47.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:47.438+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef4003690 0x7f3ef4007f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:36:47.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:47.438+0000 7f3f03fff640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef400ba50 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:47.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:47.438+0000 7f3f03fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 0x7f3ef40091b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:36:47.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:47.438+0000 7f3f03fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f1415c730 con 0x7f3ef4006620 2024-04-22T06:36:47.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:47.438+0000 7f3f03fff640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415d340 con 0x7f3ef4008da0 2024-04-22T06:36:47.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:47.438+0000 7f3f1259c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 msgr2=0x7f3ef40091b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:36:47.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:47.438+0000 7f3f1259c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 0x7f3ef40091b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:36:47.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:47.438+0000 7f3f11d9b640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef4006a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:36:47.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:47.639+0000 7f3f1259c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 msgr2=0x7f3ef40091b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:36:47.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:47.639+0000 7f3f1259c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 0x7f3ef40091b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:36:47.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:47.639+0000 7f3f11d9b640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef4006a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:36:48.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:48.040+0000 7f3f1259c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 msgr2=0x7f3ef40091b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:36:48.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:48.040+0000 7f3f1259c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 0x7f3ef40091b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:36:48.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:48.040+0000 7f3f11d9b640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef4006a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:36:48.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:48.841+0000 7f3f1259c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 msgr2=0x7f3ef40091b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:36:48.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:48.841+0000 7f3f1259c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 0x7f3ef40091b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:36:48.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:48.841+0000 7f3f11d9b640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef4006a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:36:50.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:50.439+0000 7f3f03fff640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 msgr2=0x7f3ef40091b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:50.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:50.439+0000 7f3f03fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 0x7f3ef40091b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:36:50.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:50.439+0000 7f3f03fff640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef4006a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:50.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:50.439+0000 7f3f12d9d640 1 -- reap_dead start 2024-04-22T06:36:50.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:50.439+0000 7f3f03fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 0x7f3ef40091b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:36:50.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:50.439+0000 7f3f03fff640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f140c9e40 con 0x7f3ef400ba50 2024-04-22T06:36:50.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:50.439+0000 7f3f03fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f1415c730 con 0x7f3ef4006620 2024-04-22T06:36:50.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:50.439+0000 7f3f03fff640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415d340 con 0x7f3ef4008da0 2024-04-22T06:36:50.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:50.439+0000 7f3f11d9b640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 msgr2=0x7f3ef40091b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:36:50.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:50.439+0000 7f3f1259c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef400ba50 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:50.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:50.439+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 0x7f3ef40091b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:36:50.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:50.439+0000 7f3f12d9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef4006a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:36:50.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:50.640+0000 7f3f11d9b640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 msgr2=0x7f3ef40091b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:36:50.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:50.640+0000 7f3f1259c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef400ba50 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:50.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:50.640+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 0x7f3ef40091b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:36:50.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:50.640+0000 7f3f12d9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef4006a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:36:51.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:51.041+0000 7f3f1259c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef400ba50 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:51.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:51.041+0000 7f3f11d9b640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 msgr2=0x7f3ef40091b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:36:51.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:51.041+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 0x7f3ef40091b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:36:51.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:51.041+0000 7f3f12d9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef4006a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:36:51.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:51.842+0000 7f3f1259c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef400ba50 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:51.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:51.842+0000 7f3f11d9b640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 msgr2=0x7f3ef40091b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:36:51.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:51.842+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 0x7f3ef40091b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:36:51.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:51.842+0000 7f3f12d9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef4006a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:36:53.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:53.439+0000 7f3f03fff640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 msgr2=0x7f3ef40091b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:53.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:53.439+0000 7f3f03fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4008da0 0x7f3ef40091b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:36:53.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:53.439+0000 7f3f03fff640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef4006a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:53.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:53.439+0000 7f3f03fff640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef400ba50 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:53.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:53.439+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 0x7f3ef4007f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:36:53.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:53.439+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400b0e0 0x7f3ef4007180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:36:53.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:53.439+0000 7f3f03fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415d340 con 0x7f3ef400b0e0 2024-04-22T06:36:53.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:53.439+0000 7f3f03fff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3f1415c450 con 0x7f3ef4003690 2024-04-22T06:36:53.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:53.439+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400b0e0 msgr2=0x7f3ef4007180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:36:53.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:53.439+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 msgr2=0x7f3ef4007f80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:36:53.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:53.439+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400b0e0 0x7f3ef4007180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:36:53.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:53.439+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 0x7f3ef4007f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:36:53.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:53.640+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400b0e0 msgr2=0x7f3ef4007180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:36:53.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:53.640+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 msgr2=0x7f3ef4007f80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:36:53.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:53.640+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400b0e0 0x7f3ef4007180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:36:53.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:53.640+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 0x7f3ef4007f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:36:54.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:54.041+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400b0e0 msgr2=0x7f3ef4007180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:36:54.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:54.041+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 msgr2=0x7f3ef4007f80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:36:54.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:54.041+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400b0e0 0x7f3ef4007180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:36:54.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:54.041+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 0x7f3ef4007f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:36:54.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:54.842+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400b0e0 msgr2=0x7f3ef4007180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:36:54.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:54.842+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 msgr2=0x7f3ef4007f80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:36:54.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:54.842+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400b0e0 0x7f3ef4007180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:36:54.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:54.842+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 0x7f3ef4007f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:36:56.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:56.439+0000 7f3f03fff640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 msgr2=0x7f3ef4007f80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:56.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:56.439+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4003690 0x7f3ef4007f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:36:56.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:56.439+0000 7f3f03fff640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400b0e0 msgr2=0x7f3ef4007180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:56.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:56.439+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400b0e0 0x7f3ef4007180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:36:56.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:56.439+0000 7f3f12d9d640 1 -- reap_dead start 2024-04-22T06:36:56.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:56.439+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400b0e0 0x7f3ef4007180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:36:56.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:56.439+0000 7f3f03fff640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ef4010cb0 con 0x7f3ef400ba50 2024-04-22T06:36:56.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:56.439+0000 7f3f03fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f140c9e40 con 0x7f3ef4006620 2024-04-22T06:36:56.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:56.439+0000 7f3f03fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415d340 con 0x7f3ef400b0e0 2024-04-22T06:36:56.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:56.439+0000 7f3f11d9b640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef400ba50 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:56.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:56.439+0000 7f3f12d9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400b0e0 msgr2=0x7f3ef4007180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:36:56.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:56.439+0000 7f3f1259c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef4006a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:36:56.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:56.439+0000 7f3f12d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400b0e0 0x7f3ef4007180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:36:56.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:56.640+0000 7f3f11d9b640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef400ba50 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:56.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:56.640+0000 7f3f12d9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400b0e0 msgr2=0x7f3ef4007180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:36:56.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:56.640+0000 7f3f1259c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef4006a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:36:56.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:56.640+0000 7f3f12d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400b0e0 0x7f3ef4007180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:36:57.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:57.041+0000 7f3f11d9b640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef400ba50 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:57.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:57.041+0000 7f3f1259c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef4006a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:36:57.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:57.041+0000 7f3f12d9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400b0e0 msgr2=0x7f3ef4007180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:36:57.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:57.041+0000 7f3f12d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400b0e0 0x7f3ef4007180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:36:57.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:57.842+0000 7f3f11d9b640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef400ba50 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:36:57.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:57.842+0000 7f3f1259c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef4006a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:36:57.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:57.842+0000 7f3f12d9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400b0e0 msgr2=0x7f3ef4007180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:36:57.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:57.842+0000 7f3f12d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400b0e0 0x7f3ef4007180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:36:59.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:59.440+0000 7f3f03fff640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400b0e0 msgr2=0x7f3ef4007180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:59.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:59.440+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400b0e0 0x7f3ef4007180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:36:59.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:59.440+0000 7f3f03fff640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef4006a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:59.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:59.440+0000 7f3f03fff640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef400ba50 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:36:59.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:59.440+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 0x7f3ef4007f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:36:59.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:59.440+0000 7f3f03fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4003690 0x7f3ef4008da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:36:59.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:59.440+0000 7f3f03fff640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415d340 con 0x7f3ef4003690 2024-04-22T06:36:59.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:59.440+0000 7f3f03fff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3f1415c450 con 0x7f3ef4005ae0 2024-04-22T06:36:59.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:59.440+0000 7f3f11d9b640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4003690 msgr2=0x7f3ef4008da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:36:59.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:59.440+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4003690 0x7f3ef4008da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:36:59.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:59.440+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 msgr2=0x7f3ef4007f80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:36:59.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:59.440+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 0x7f3ef4007f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:36:59.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:59.641+0000 7f3f11d9b640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4003690 msgr2=0x7f3ef4008da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:36:59.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:59.641+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4003690 0x7f3ef4008da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:36:59.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:59.641+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 msgr2=0x7f3ef4007f80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:36:59.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:36:59.641+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 0x7f3ef4007f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:37:00.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:00.042+0000 7f3f11d9b640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4003690 msgr2=0x7f3ef4008da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:37:00.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:00.042+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4003690 0x7f3ef4008da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:37:00.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:00.042+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 msgr2=0x7f3ef4007f80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:37:00.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:00.042+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 0x7f3ef4007f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:37:00.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:00.843+0000 7f3f11d9b640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4003690 msgr2=0x7f3ef4008da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:37:00.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:00.843+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4003690 0x7f3ef4008da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:37:00.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:00.843+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 msgr2=0x7f3ef4007f80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:37:00.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:00.843+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 0x7f3ef4007f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:37:02.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:02.440+0000 7f3f03fff640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 msgr2=0x7f3ef4007f80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:37:02.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:02.440+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 0x7f3ef4007f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:37:02.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:02.440+0000 7f3f03fff640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4003690 msgr2=0x7f3ef4008da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:37:02.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:02.440+0000 7f3f03fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4003690 0x7f3ef4008da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:37:02.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:02.440+0000 7f3f12d9d640 1 -- reap_dead start 2024-04-22T06:37:02.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:02.440+0000 7f3f03fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef400b0e0 0x7f3ef4007f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:37:02.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:02.440+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef40105a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:37:02.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:02.440+0000 7f3f03fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f1415c730 con 0x7f3ef4006620 2024-04-22T06:37:02.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:02.440+0000 7f3f03fff640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415d340 con 0x7f3ef400b0e0 2024-04-22T06:37:02.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:02.440+0000 7f3f03fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415c450 con 0x7f3ef400ba50 2024-04-22T06:37:02.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:02.440+0000 7f3f11d9b640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef400b0e0 msgr2=0x7f3ef4007f80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:37:02.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:02.440+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef400b0e0 0x7f3ef4007f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:37:02.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:02.440+0000 7f3f1259c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef4006a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:37:02.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:02.440+0000 7f3f12d9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 msgr2=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:37:02.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:02.440+0000 7f3f12d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef40105a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:37:02.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:02.641+0000 7f3f11d9b640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef400b0e0 msgr2=0x7f3ef4007f80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:37:02.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:02.641+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef400b0e0 0x7f3ef4007f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:37:02.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:02.641+0000 7f3f12d9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 msgr2=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:37:02.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:02.641+0000 7f3f1259c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef4006a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:37:02.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:02.641+0000 7f3f12d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef40105a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:37:03.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:03.042+0000 7f3f11d9b640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef400b0e0 msgr2=0x7f3ef4007f80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:37:03.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:03.042+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef400b0e0 0x7f3ef4007f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:37:03.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:03.042+0000 7f3f1259c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef4006a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:37:03.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:03.042+0000 7f3f12d9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 msgr2=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:37:03.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:03.042+0000 7f3f12d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef40105a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:37:03.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:03.843+0000 7f3f11d9b640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef400b0e0 msgr2=0x7f3ef4007f80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:37:03.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:03.843+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef400b0e0 0x7f3ef4007f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:37:03.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:03.843+0000 7f3f1259c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef4006a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:37:03.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:03.843+0000 7f3f12d9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 msgr2=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:37:03.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:03.843+0000 7f3f12d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef40105a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:37:05.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:05.440+0000 7f3f03fff640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 msgr2=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:37:05.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:05.440+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef40105a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:37:05.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:05.440+0000 7f3f03fff640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef400b0e0 msgr2=0x7f3ef4007f80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:37:05.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:05.440+0000 7f3f03fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef400b0e0 0x7f3ef4007f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:37:05.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:05.440+0000 7f3f03fff640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef4006a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:37:05.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:05.440+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 0x7f3ef4008d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:37:05.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:05.440+0000 7f3f03fff640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ef4010cb0 con 0x7f3ef40092a0 2024-04-22T06:37:05.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:05.440+0000 7f3f03fff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3f1415d340 con 0x7f3ef4005ae0 2024-04-22T06:37:05.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:05.440+0000 7f3f11d9b640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef40092a0 legacy=0x7f3ef4007180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:37:05.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:05.440+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 msgr2=0x7f3ef4008d60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:37:05.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:05.440+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 0x7f3ef4008d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:37:05.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:05.641+0000 7f3f11d9b640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef40092a0 legacy=0x7f3ef4007180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:37:05.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:05.641+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 msgr2=0x7f3ef4008d60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:37:05.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:05.641+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 0x7f3ef4008d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:37:06.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:06.042+0000 7f3f11d9b640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef40092a0 legacy=0x7f3ef4007180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:37:06.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:06.042+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 msgr2=0x7f3ef4008d60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:37:06.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:06.042+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 0x7f3ef4008d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:37:06.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:06.842+0000 7f3f11d9b640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef40092a0 legacy=0x7f3ef4007180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:37:06.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:06.842+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 msgr2=0x7f3ef4008d60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:37:06.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:06.842+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 0x7f3ef4008d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:37:08.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:08.439+0000 7f3f03fff640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 msgr2=0x7f3ef4008d60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:37:08.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:08.439+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 0x7f3ef4008d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:37:08.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:08.439+0000 7f3f03fff640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef40092a0 legacy=0x7f3ef4007180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:37:08.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:08.439+0000 7f3f12d9d640 1 -- reap_dead start 2024-04-22T06:37:08.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:08.439+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef40105a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:37:08.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:08.439+0000 7f3f03fff640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f1415c730 con 0x7f3ef40092a0 2024-04-22T06:37:08.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:08.439+0000 7f3f03fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ef4010cb0 con 0x7f3ef4006620 2024-04-22T06:37:08.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:08.439+0000 7f3f03fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415d340 con 0x7f3ef400ba50 2024-04-22T06:37:08.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:08.439+0000 7f3f11d9b640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef40092a0 legacy=0x7f3ef4007180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:37:08.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:08.439+0000 7f3f12d9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 msgr2=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:37:08.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:08.439+0000 7f3f1259c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef4006a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:37:08.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:08.439+0000 7f3f12d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef40105a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:37:08.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:08.640+0000 7f3f11d9b640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef40092a0 legacy=0x7f3ef4007180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:37:08.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:08.640+0000 7f3f1259c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef4006a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:37:08.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:08.640+0000 7f3f12d9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 msgr2=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:37:08.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:08.640+0000 7f3f12d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef40105a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:37:09.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:09.041+0000 7f3f11d9b640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef40092a0 legacy=0x7f3ef4007180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:37:09.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:09.041+0000 7f3f1259c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef4006a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:37:09.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:09.041+0000 7f3f12d9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 msgr2=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:37:09.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:09.041+0000 7f3f12d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef40105a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:37:09.600 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T06:37:09.603 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T06:37:09.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:09.842+0000 7f3f11d9b640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef40092a0 legacy=0x7f3ef4007180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:37:09.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:09.842+0000 7f3f12d9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 msgr2=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:37:09.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:09.842+0000 7f3f1259c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef4006a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:37:09.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:09.842+0000 7f3f12d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef40105a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:37:11.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:11.440+0000 7f3f03fff640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 msgr2=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:37:11.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:11.440+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef40105a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:37:11.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:11.440+0000 7f3f03fff640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef4006a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:37:11.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:11.440+0000 7f3f03fff640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef40092a0 legacy=0x7f3ef4007180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:37:11.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:11.440+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 0x7f3ef4008d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:37:11.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:11.440+0000 7f3f03fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007f80 0x7f3ef401b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:37:11.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:11.440+0000 7f3f03fff640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415d340 con 0x7f3ef4007f80 2024-04-22T06:37:11.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:11.440+0000 7f3f03fff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3f1415c450 con 0x7f3ef400b0e0 2024-04-22T06:37:11.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:11.440+0000 7f3f11d9b640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007f80 msgr2=0x7f3ef401b9e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:37:11.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:11.440+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007f80 0x7f3ef401b9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:37:11.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:11.440+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 msgr2=0x7f3ef4008d60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:37:11.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:11.440+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 0x7f3ef4008d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:37:11.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:11.641+0000 7f3f11d9b640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007f80 msgr2=0x7f3ef401b9e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:37:11.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:11.641+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007f80 0x7f3ef401b9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:37:11.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:11.641+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 msgr2=0x7f3ef4008d60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:37:11.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:11.641+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 0x7f3ef4008d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:37:12.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:12.042+0000 7f3f11d9b640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007f80 msgr2=0x7f3ef401b9e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:37:12.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:12.042+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007f80 0x7f3ef401b9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:37:12.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:12.042+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 msgr2=0x7f3ef4008d60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:37:12.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:12.042+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 0x7f3ef4008d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:37:12.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:12.843+0000 7f3f11d9b640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007f80 msgr2=0x7f3ef401b9e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:37:12.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:12.843+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007f80 0x7f3ef401b9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:37:12.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:12.843+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 msgr2=0x7f3ef4008d60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:37:12.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:12.843+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 0x7f3ef4008d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:37:14.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:14.440+0000 7f3f03fff640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 msgr2=0x7f3ef4008d60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:37:14.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:14.440+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 0x7f3ef4008d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:37:14.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:14.440+0000 7f3f03fff640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007f80 msgr2=0x7f3ef401b9e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:37:14.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:14.440+0000 7f3f03fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007f80 0x7f3ef401b9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:37:14.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:14.440+0000 7f3f12d9d640 1 -- reap_dead start 2024-04-22T06:37:14.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:14.440+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef40105a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:37:14.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:14.440+0000 7f3f03fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007f80 0x7f3ef401b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:37:14.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:14.440+0000 7f3f03fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f140c9e40 con 0x7f3ef4006620 2024-04-22T06:37:14.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:14.440+0000 7f3f03fff640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415d340 con 0x7f3ef4007f80 2024-04-22T06:37:14.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:14.440+0000 7f3f03fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415c450 con 0x7f3ef400ba50 2024-04-22T06:37:14.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:14.440+0000 7f3f12d9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007f80 msgr2=0x7f3ef401b9e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:37:14.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:14.440+0000 7f3f12d9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007f80 0x7f3ef401b9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:37:14.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:14.440+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 msgr2=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:37:14.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:14.440+0000 7f3f1259c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef4006a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:37:14.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:14.440+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef40105a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:37:14.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:14.641+0000 7f3f12d9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007f80 msgr2=0x7f3ef401b9e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:37:14.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:14.641+0000 7f3f12d9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007f80 0x7f3ef401b9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:37:14.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:14.641+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 msgr2=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:37:14.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:14.641+0000 7f3f1259c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef4006a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:37:14.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:14.641+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef40105a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:37:15.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:15.042+0000 7f3f12d9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007f80 msgr2=0x7f3ef401b9e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:37:15.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:15.042+0000 7f3f12d9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007f80 0x7f3ef401b9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:37:15.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:15.042+0000 7f3f1259c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef4006a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:37:15.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:15.042+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 msgr2=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:37:15.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:15.042+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef40105a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:37:15.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:15.843+0000 7f3f12d9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007f80 msgr2=0x7f3ef401b9e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:37:15.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:15.843+0000 7f3f12d9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007f80 0x7f3ef401b9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:37:15.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:15.843+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 msgr2=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:37:15.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:15.843+0000 7f3f1259c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef4006a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:37:15.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:15.843+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef40105a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:37:17.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:17.441+0000 7f3f03fff640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 msgr2=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:37:17.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:17.441+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef40105a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:37:17.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:17.441+0000 7f3f03fff640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007f80 msgr2=0x7f3ef401b9e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:37:17.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:17.441+0000 7f3f03fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007f80 0x7f3ef401b9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:37:17.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:17.441+0000 7f3f03fff640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef4006a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:37:17.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:17.441+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 0x7f3ef4008d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:37:17.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:17.441+0000 7f3f03fff640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f1415c730 con 0x7f3ef4009280 2024-04-22T06:37:17.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:17.441+0000 7f3f03fff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3f1415d340 con 0x7f3ef400b0e0 2024-04-22T06:37:17.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:17.441+0000 7f3f11d9b640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef4009280 legacy=0x7f3ef4007180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:37:17.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:17.441+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 msgr2=0x7f3ef4008d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:37:17.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:17.441+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 0x7f3ef4008d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:37:17.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:17.642+0000 7f3f11d9b640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef4009280 legacy=0x7f3ef4007180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:37:17.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:17.642+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 msgr2=0x7f3ef4008d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:37:17.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:17.642+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 0x7f3ef4008d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:37:18.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:18.043+0000 7f3f11d9b640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef4009280 legacy=0x7f3ef4007180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:37:18.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:18.043+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 msgr2=0x7f3ef4008d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:37:18.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:18.043+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 0x7f3ef4008d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:37:18.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:18.844+0000 7f3f11d9b640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef4009280 legacy=0x7f3ef4007180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:37:18.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:18.844+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 msgr2=0x7f3ef4008d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:37:18.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:18.844+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 0x7f3ef4008d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:37:20.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:20.441+0000 7f3f03fff640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 msgr2=0x7f3ef4008d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:37:20.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:20.441+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 0x7f3ef4008d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:37:20.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:20.441+0000 7f3f03fff640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef4009280 legacy=0x7f3ef4007180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:37:20.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:20.441+0000 7f3f12d9d640 1 -- reap_dead start 2024-04-22T06:37:20.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:20.441+0000 7f3f03fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007f80 0x7f3ef401b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:37:20.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:20.441+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef40105a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:37:20.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:20.441+0000 7f3f03fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f140c9e40 con 0x7f3ef4006620 2024-04-22T06:37:20.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:20.441+0000 7f3f03fff640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415d340 con 0x7f3ef4007f80 2024-04-22T06:37:20.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:20.441+0000 7f3f03fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415c450 con 0x7f3ef400ba50 2024-04-22T06:37:20.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:20.441+0000 7f3f1259c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007f80 msgr2=0x7f3ef401b9e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:37:20.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:20.441+0000 7f3f1259c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007f80 0x7f3ef401b9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:37:20.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:20.441+0000 7f3f11d9b640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef4006a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:37:20.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:20.441+0000 7f3f12d9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 msgr2=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:37:20.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:20.441+0000 7f3f12d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef40105a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:37:20.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:20.642+0000 7f3f1259c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007f80 msgr2=0x7f3ef401b9e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:37:20.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:20.642+0000 7f3f1259c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007f80 0x7f3ef401b9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:37:20.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:20.642+0000 7f3f12d9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 msgr2=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:37:20.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:20.642+0000 7f3f11d9b640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef4006a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:37:20.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:20.642+0000 7f3f12d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef40105a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:37:21.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:21.043+0000 7f3f1259c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007f80 msgr2=0x7f3ef401b9e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:37:21.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:21.043+0000 7f3f1259c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007f80 0x7f3ef401b9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:37:21.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:21.043+0000 7f3f12d9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 msgr2=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:37:21.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:21.043+0000 7f3f11d9b640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef4006a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:37:21.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:21.043+0000 7f3f12d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef40105a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:37:21.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:21.844+0000 7f3f1259c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007f80 msgr2=0x7f3ef401b9e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:37:21.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:21.844+0000 7f3f1259c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007f80 0x7f3ef401b9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:37:21.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:21.844+0000 7f3f12d9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 msgr2=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:37:21.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:21.844+0000 7f3f11d9b640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef4006a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:37:21.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:21.844+0000 7f3f12d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef40105a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:37:23.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:23.441+0000 7f3f03fff640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 msgr2=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:37:23.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:23.441+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef40105a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:37:23.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:23.441+0000 7f3f03fff640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007f80 msgr2=0x7f3ef401b9e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:37:23.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:23.441+0000 7f3f03fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4007f80 0x7f3ef401b9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:37:23.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:23.441+0000 7f3f03fff640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef4006a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:37:23.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:23.441+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 0x7f3ef4008d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:37:23.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:23.441+0000 7f3f03fff640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f1415c730 con 0x7f3ef4009240 2024-04-22T06:37:23.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:23.441+0000 7f3f03fff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3f1415d340 con 0x7f3ef4005ae0 2024-04-22T06:37:23.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:23.441+0000 7f3f11d9b640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef4009240 legacy=0x7f3ef4007180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:37:23.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:23.441+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 msgr2=0x7f3ef4008d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:37:23.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:23.442+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 0x7f3ef4008d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:37:23.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:23.642+0000 7f3f11d9b640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef4009240 legacy=0x7f3ef4007180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:37:23.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:23.643+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 msgr2=0x7f3ef4008d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:37:23.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:23.643+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 0x7f3ef4008d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:37:24.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:24.043+0000 7f3f11d9b640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef4009240 legacy=0x7f3ef4007180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:37:24.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:24.044+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 msgr2=0x7f3ef4008d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:37:24.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:24.044+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 0x7f3ef4008d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:37:24.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:24.844+0000 7f3f11d9b640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef4009240 legacy=0x7f3ef4007180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:37:24.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:24.845+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 msgr2=0x7f3ef4008d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:37:24.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:24.845+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 0x7f3ef4008d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:37:26.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:26.442+0000 7f3f03fff640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 msgr2=0x7f3ef4008d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:37:26.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:26.442+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef4005ae0 0x7f3ef4008d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:37:26.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:26.442+0000 7f3f03fff640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef4009240 legacy=0x7f3ef4007180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:37:26.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:26.442+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 0x7f3ef401df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:37:26.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:26.442+0000 7f3f12d9d640 1 -- reap_dead start 2024-04-22T06:37:26.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:26.442+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef4008d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:37:26.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:26.442+0000 7f3f03fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f140c9e40 con 0x7f3ef4006620 2024-04-22T06:37:26.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:26.442+0000 7f3f03fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415d340 con 0x7f3ef400ba50 2024-04-22T06:37:26.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:26.442+0000 7f3f03fff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3f1415c450 con 0x7f3ef400b0e0 2024-04-22T06:37:26.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:26.442+0000 7f3f12d9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:37:26.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:26.442+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 msgr2=0x7f3ef401df30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:37:26.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:26.442+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 msgr2=0x7f3ef4008d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:37:26.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:26.442+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef4008d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:37:26.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:26.442+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 0x7f3ef401df30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:37:26.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:26.643+0000 7f3f12d9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:37:26.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:26.643+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 msgr2=0x7f3ef4008d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:37:26.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:26.643+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 msgr2=0x7f3ef401df30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:37:26.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:26.643+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef4008d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:37:26.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:26.643+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 0x7f3ef401df30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:37:27.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:27.044+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 msgr2=0x7f3ef401df30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:37:27.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:27.044+0000 7f3f12d9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:37:27.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:27.044+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 msgr2=0x7f3ef4008d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:37:27.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:27.044+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 0x7f3ef401df30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:37:27.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:27.044+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef4008d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:37:27.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:27.845+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 msgr2=0x7f3ef401df30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:37:27.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:27.845+0000 7f3f12d9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:37:27.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:27.845+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 0x7f3ef401df30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:37:27.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:27.845+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 msgr2=0x7f3ef4008d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:37:27.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:27.845+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef4008d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:37:29.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:29.442+0000 7f3f03fff640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 msgr2=0x7f3ef401df30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:37:29.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:29.442+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 0x7f3ef401df30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:37:29.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:29.442+0000 7f3f03fff640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 msgr2=0x7f3ef4008d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:37:29.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:29.442+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef4008d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:37:29.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:29.442+0000 7f3f03fff640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:37:29.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:29.442+0000 7f3f03fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009240 0x7f3ef40135b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:37:29.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:29.442+0000 7f3f03fff640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f1415c730 con 0x7f3ef4007f80 2024-04-22T06:37:29.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:29.442+0000 7f3f03fff640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415d340 con 0x7f3ef4009240 2024-04-22T06:37:29.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:29.442+0000 7f3f11d9b640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef4007f80 legacy=0x7f3ef4013af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:37:29.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:29.442+0000 7f3f1259c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009240 msgr2=0x7f3ef40135b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:37:29.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:29.442+0000 7f3f1259c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009240 0x7f3ef40135b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:37:29.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:29.643+0000 7f3f11d9b640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef4007f80 legacy=0x7f3ef4013af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:37:29.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:29.643+0000 7f3f1259c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009240 msgr2=0x7f3ef40135b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:37:29.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:29.643+0000 7f3f1259c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009240 0x7f3ef40135b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:37:30.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:30.044+0000 7f3f1259c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009240 msgr2=0x7f3ef40135b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:37:30.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:30.044+0000 7f3f11d9b640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef4007f80 legacy=0x7f3ef4013af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:37:30.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:30.044+0000 7f3f1259c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009240 0x7f3ef40135b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:37:30.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:30.845+0000 7f3f11d9b640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef4007f80 legacy=0x7f3ef4013af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:37:30.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:30.845+0000 7f3f1259c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009240 msgr2=0x7f3ef40135b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:37:30.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:30.845+0000 7f3f1259c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009240 0x7f3ef40135b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:37:32.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:32.442+0000 7f3f03fff640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009240 msgr2=0x7f3ef40135b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:37:32.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:32.442+0000 7f3f03fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009240 0x7f3ef40135b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:37:32.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:32.442+0000 7f3f03fff640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef4007f80 legacy=0x7f3ef4013af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:37:32.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:32.442+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 0x7f3ef401b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:37:32.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:32.442+0000 7f3f12d9d640 1 -- reap_dead start 2024-04-22T06:37:32.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:32.442+0000 7f3f03fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009240 0x7f3ef40135b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:37:32.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:32.442+0000 7f3f03fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f140c9e40 con 0x7f3ef4006620 2024-04-22T06:37:32.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:32.442+0000 7f3f03fff640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415d340 con 0x7f3ef4009240 2024-04-22T06:37:32.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:32.442+0000 7f3f03fff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3f1415c450 con 0x7f3ef400b0e0 2024-04-22T06:37:32.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:32.442+0000 7f3f11d9b640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009240 msgr2=0x7f3ef40135b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:37:32.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:32.442+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009240 0x7f3ef40135b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:37:32.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:32.442+0000 7f3f12d9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:37:32.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:32.442+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 msgr2=0x7f3ef401b9e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:37:32.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:32.442+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 0x7f3ef401b9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:37:32.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:32.643+0000 7f3f11d9b640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009240 msgr2=0x7f3ef40135b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:37:32.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:32.643+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009240 0x7f3ef40135b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:37:32.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:32.643+0000 7f3f12d9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:37:32.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:32.643+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 msgr2=0x7f3ef401b9e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:37:32.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:32.643+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 0x7f3ef401b9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:37:33.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:33.044+0000 7f3f11d9b640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009240 msgr2=0x7f3ef40135b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:37:33.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:33.044+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009240 0x7f3ef40135b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:37:33.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:33.044+0000 7f3f12d9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:37:33.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:33.044+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 msgr2=0x7f3ef401b9e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:37:33.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:33.044+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 0x7f3ef401b9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:37:33.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:33.845+0000 7f3f11d9b640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009240 msgr2=0x7f3ef40135b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:37:33.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:33.845+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009240 0x7f3ef40135b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:37:33.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:33.845+0000 7f3f12d9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:37:33.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:33.845+0000 7f3f1259c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 msgr2=0x7f3ef401b9e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:37:33.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:33.845+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 0x7f3ef401b9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:37:35.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:35.443+0000 7f3f03fff640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 msgr2=0x7f3ef401b9e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:37:35.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:35.443+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef400b0e0 0x7f3ef401b9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:37:35.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:35.443+0000 7f3f03fff640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009240 msgr2=0x7f3ef40135b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:37:35.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:35.443+0000 7f3f03fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009240 0x7f3ef40135b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:37:35.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:35.443+0000 7f3f03fff640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:37:35.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:35.443+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef4008d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:37:35.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:35.443+0000 7f3f03fff640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f1415c730 con 0x7f3ef4007f80 2024-04-22T06:37:35.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:35.443+0000 7f3f03fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415d340 con 0x7f3ef400ba50 2024-04-22T06:37:35.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:35.443+0000 7f3f1259c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef4007f80 legacy=0x7f3ef4013af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:37:35.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:35.443+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 msgr2=0x7f3ef4008d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:37:35.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:35.443+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef4008d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:37:35.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:35.644+0000 7f3f1259c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef4007f80 legacy=0x7f3ef4013af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:37:35.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:35.644+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 msgr2=0x7f3ef4008d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:37:35.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:35.644+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef4008d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:37:36.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:36.045+0000 7f3f1259c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef4007f80 legacy=0x7f3ef4013af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:37:36.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:36.045+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 msgr2=0x7f3ef4008d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:37:36.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:36.045+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef4008d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:37:36.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:36.846+0000 7f3f1259c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef4007f80 legacy=0x7f3ef4013af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:37:36.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:36.846+0000 7f3f11d9b640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 msgr2=0x7f3ef4008d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:37:36.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:36.846+0000 7f3f11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef4008d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:37:38.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:38.443+0000 7f3f03fff640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 msgr2=0x7f3ef4008d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:37:38.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:38.443+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef400ba50 0x7f3ef4008d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:37:38.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:38.443+0000 7f3f03fff640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef4007f80 legacy=0x7f3ef4013af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:37:38.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:38.443+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef401bf20 0x7f3ef401e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:37:38.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:38.443+0000 7f3f03fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009240 0x7f3ef40135b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:37:38.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:38.443+0000 7f3f12d9d640 1 -- reap_dead start 2024-04-22T06:37:38.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:38.443+0000 7f3f03fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f140c9e40 con 0x7f3ef4006620 2024-04-22T06:37:38.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:38.443+0000 7f3f03fff640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415d340 con 0x7f3ef4009240 2024-04-22T06:37:38.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:38.443+0000 7f3f03fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415c450 con 0x7f3ef401bf20 2024-04-22T06:37:38.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:38.443+0000 7f3f12d9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009240 msgr2=0x7f3ef40135b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:37:38.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:38.443+0000 7f3f12d9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009240 0x7f3ef40135b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:37:38.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:38.443+0000 7f3f11d9b640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:37:38.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:38.443+0000 7f3f1259c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef401bf20 msgr2=0x7f3ef401e310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:37:38.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:38.443+0000 7f3f1259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef401bf20 0x7f3ef401e310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:37:38.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:38.644+0000 7f3f12d9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009240 msgr2=0x7f3ef40135b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:37:38.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:38.644+0000 7f3f12d9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009240 0x7f3ef40135b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:37:38.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:38.644+0000 7f3f11d9b640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:37:38.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:38.644+0000 7f3f1259c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef401bf20 msgr2=0x7f3ef401e310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:37:38.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:38.644+0000 7f3f1259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef401bf20 0x7f3ef401e310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:37:39.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:39.045+0000 7f3f12d9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009240 msgr2=0x7f3ef40135b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:37:39.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:39.045+0000 7f3f12d9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009240 0x7f3ef40135b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:37:39.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:39.045+0000 7f3f1259c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef401bf20 msgr2=0x7f3ef401e310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:37:39.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:39.045+0000 7f3f11d9b640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:37:39.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:39.045+0000 7f3f1259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef401bf20 0x7f3ef401e310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:37:39.736 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T06:37:39.740 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T06:37:39.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:39.846+0000 7f3f12d9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009240 msgr2=0x7f3ef40135b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T06:37:39.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:39.846+0000 7f3f12d9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009240 0x7f3ef40135b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:37:39.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:39.846+0000 7f3f1259c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef401bf20 msgr2=0x7f3ef401e310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T06:37:39.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:39.846+0000 7f3f11d9b640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T06:37:39.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:39.846+0000 7f3f1259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef401bf20 0x7f3ef401e310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:37:41.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:41.443+0000 7f3f03fff640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f3ef401bf20 msgr2=0x7f3ef401e310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:37:41.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:41.443+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ef401bf20 0x7f3ef401e310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:37:41.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:41.443+0000 7f3f03fff640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009240 msgr2=0x7f3ef40135b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:37:41.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:41.443+0000 7f3f03fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3ef4009240 0x7f3ef40135b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T06:37:41.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:41.443+0000 7f3f03fff640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f3ef4006620 legacy=0x7f3ef40105a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T06:37:41.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:41.443+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef401b9e0 0x7f3ef4008d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T06:37:41.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:41.443+0000 7f3f03fff640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f1415c730 con 0x7f3ef400b0e0 2024-04-22T06:37:41.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:41.443+0000 7f3f03fff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3f1415d340 con 0x7f3ef401b9e0 2024-04-22T06:37:41.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:41.443+0000 7f3f1259c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef400b0e0 legacy=0x7f3ef4013af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:37:41.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:41.443+0000 7f3f11d9b640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef401b9e0 msgr2=0x7f3ef4008d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:37:41.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:41.443+0000 7f3f11d9b640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef401b9e0 0x7f3ef4008d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T06:37:41.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:41.644+0000 7f3f1259c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef400b0e0 legacy=0x7f3ef4013af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:37:41.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:41.644+0000 7f3f11d9b640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef401b9e0 msgr2=0x7f3ef4008d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:37:41.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:41.644+0000 7f3f11d9b640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef401b9e0 0x7f3ef4008d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T06:37:42.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:42.045+0000 7f3f1259c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef400b0e0 legacy=0x7f3ef4013af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:37:42.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:42.045+0000 7f3f11d9b640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef401b9e0 msgr2=0x7f3ef4008d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:37:42.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:42.045+0000 7f3f11d9b640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef401b9e0 0x7f3ef4008d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T06:37:42.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:42.846+0000 7f3f1259c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3ef400b0e0 legacy=0x7f3ef4013af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T06:37:42.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:42.846+0000 7f3f11d9b640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef401b9e0 msgr2=0x7f3ef4008d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T06:37:42.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T06:37:42.846+0000 7f3f11d9b640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ef401b9e0 0x7f3ef4008d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T06:37:44.355 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-04-22T06:37:44.356 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1918, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 310, in task stop_all_quiescers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 260, in stop_all_quiescers thrasher.stop() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 252, in stop self.tell_quiesce_leader( "--cancel", "--all" ) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 110, in tell_quiesce_leader leader = self.fs.get_var('qdb_leader') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 959, in get_var return self.get_mds_map(status=status)[var] File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 955, in get_mds_map status = self.status() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 408, in status return FSStatus(self.mon_manager, epoch) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 88, in __init__ self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout return self.run_ceph_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd return self.mon_manager.run_cluster_cmd(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2822, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2654, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2420, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1656, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2024-04-22T06:37:44.357 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-22T06:37:44.358 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-22T06:37:44.358 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T06:37:44.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T06:37:44.358 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1918, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 310, in task stop_all_quiescers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 260, in stop_all_quiescers thrasher.stop() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 252, in stop self.tell_quiesce_leader( "--cancel", "--all" ) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 110, in tell_quiesce_leader leader = self.fs.get_var('qdb_leader') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 959, in get_var return self.get_mds_map(status=status)[var] File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 955, in get_mds_map status = self.status() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 408, in status return FSStatus(self.mon_manager, epoch) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 88, in __init__ self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout return self.run_ceph_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd return self.mon_manager.run_cluster_cmd(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1447, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2822, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2654, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2420, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1656, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a' 2024-04-22T06:37:44.359 INFO:tasks.ceph.mds.a:Stopped 2024-04-22T06:37:44.359 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-22T06:37:44.359 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T06:37:44.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T06:37:44.359 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1918, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 310, in task stop_all_quiescers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 260, in stop_all_quiescers thrasher.stop() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 252, in stop self.tell_quiesce_leader( "--cancel", "--all" ) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 110, in tell_quiesce_leader leader = self.fs.get_var('qdb_leader') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 959, in get_var return self.get_mds_map(status=status)[var] File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 955, in get_mds_map status = self.status() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 408, in status return FSStatus(self.mon_manager, epoch) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 88, in __init__ self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout return self.run_ceph_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd return self.mon_manager.run_cluster_cmd(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1447, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2822, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2654, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2420, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1656, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c' 2024-04-22T06:37:44.360 INFO:tasks.ceph.mds.c:Stopped 2024-04-22T06:37:44.360 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-04-22T06:37:44.360 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T06:37:44.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T06:37:44.360 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1918, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 310, in task stop_all_quiescers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 260, in stop_all_quiescers thrasher.stop() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 252, in stop self.tell_quiesce_leader( "--cancel", "--all" ) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 110, in tell_quiesce_leader leader = self.fs.get_var('qdb_leader') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 959, in get_var return self.get_mds_map(status=status)[var] File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 955, in get_mds_map status = self.status() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 408, in status return FSStatus(self.mon_manager, epoch) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 88, in __init__ self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout return self.run_ceph_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd return self.mon_manager.run_cluster_cmd(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1447, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2822, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2654, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2420, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1656, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e' 2024-04-22T06:37:44.361 INFO:tasks.ceph.mds.e:Stopped 2024-04-22T06:37:44.361 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-22T06:37:44.361 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T06:37:44.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T06:37:44.361 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1918, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 310, in task stop_all_quiescers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 260, in stop_all_quiescers thrasher.stop() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 252, in stop self.tell_quiesce_leader( "--cancel", "--all" ) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 110, in tell_quiesce_leader leader = self.fs.get_var('qdb_leader') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 959, in get_var return self.get_mds_map(status=status)[var] File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 955, in get_mds_map status = self.status() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 408, in status return FSStatus(self.mon_manager, epoch) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 88, in __init__ self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout return self.run_ceph_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd return self.mon_manager.run_cluster_cmd(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1447, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2822, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2654, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2420, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1656, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi070 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b' 2024-04-22T06:37:44.361 INFO:tasks.ceph.mds.b:Stopped 2024-04-22T06:37:44.362 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-22T06:37:44.362 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T06:37:44.362 INFO:tasks.ceph.mds.d:Stopped 2024-04-22T06:37:44.362 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2024-04-22T06:37:44.362 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T06:37:44.362 INFO:tasks.ceph.mds.f:Stopped 2024-04-22T06:37:44.362 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-22T06:37:44.363 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-22T06:37:44.363 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T06:37:44.363 INFO:tasks.ceph.osd.0:Stopped 2024-04-22T06:37:44.363 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-22T06:37:44.363 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T06:37:44.363 INFO:tasks.ceph.osd.1:Stopped 2024-04-22T06:37:44.363 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-22T06:37:44.363 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T06:37:44.363 INFO:tasks.ceph.osd.2:Stopped 2024-04-22T06:37:44.364 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-22T06:37:44.364 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T06:37:44.364 INFO:tasks.ceph.osd.3:Stopped 2024-04-22T06:37:44.364 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-22T06:37:44.364 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T06:37:44.364 INFO:tasks.ceph.osd.4:Stopped 2024-04-22T06:37:44.364 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-22T06:37:44.364 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T06:37:44.364 INFO:tasks.ceph.osd.5:Stopped 2024-04-22T06:37:44.365 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-22T06:37:44.365 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T06:37:44.365 INFO:tasks.ceph.osd.6:Stopped 2024-04-22T06:37:44.365 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-22T06:37:44.365 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T06:37:44.365 INFO:tasks.ceph.osd.7:Stopped 2024-04-22T06:37:44.365 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-22T06:37:44.366 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-22T06:37:44.366 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T06:37:44.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T06:37:44.366 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1918, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 310, in task stop_all_quiescers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 260, in stop_all_quiescers thrasher.stop() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 252, in stop self.tell_quiesce_leader( "--cancel", "--all" ) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 110, in tell_quiesce_leader leader = self.fs.get_var('qdb_leader') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 959, in get_var return self.get_mds_map(status=status)[var] File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 955, in get_mds_map status = self.status() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 408, in status return FSStatus(self.mon_manager, epoch) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 88, in __init__ self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout return self.run_ceph_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd return self.mon_manager.run_cluster_cmd(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1447, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2822, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2654, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2420, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1656, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y' 2024-04-22T06:37:44.366 INFO:tasks.ceph.mgr.y:Stopped 2024-04-22T06:37:44.366 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-22T06:37:44.366 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T06:37:44.367 INFO:tasks.ceph.mgr.x:Stopped 2024-04-22T06:37:44.367 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-22T06:37:44.367 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-22T06:37:44.367 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T06:37:44.367 INFO:tasks.ceph.mon.a:Stopped 2024-04-22T06:37:44.367 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-22T06:37:44.368 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T06:37:44.368 INFO:tasks.ceph.mon.b:Stopped 2024-04-22T06:37:44.368 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-22T06:37:44.368 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T06:37:44.368 INFO:tasks.ceph.mon.c:Stopped 2024-04-22T06:37:44.368 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-22T06:37:44.369 DEBUG:teuthology.orchestra.run.smithi044:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | head -n 1 2024-04-22T06:37:44.414 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi044.front.sepia.ceph.com 2024-04-22T06:37:44.414 DEBUG:teuthology.orchestra.run.smithi044:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-22T06:37:44.546 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi044.front.sepia.ceph.com 2024-04-22T06:37:44.546 DEBUG:teuthology.orchestra.run.smithi044:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-22T06:37:44.611 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi044.front.sepia.ceph.com 2024-04-22T06:37:44.611 DEBUG:teuthology.orchestra.run.smithi044:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-22T06:37:44.679 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi044.front.sepia.ceph.com 2024-04-22T06:37:44.679 DEBUG:teuthology.orchestra.run.smithi044:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-22T06:37:44.735 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi070.front.sepia.ceph.com 2024-04-22T06:37:44.735 DEBUG:teuthology.orchestra.run.smithi070:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-22T06:37:44.870 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi070.front.sepia.ceph.com 2024-04-22T06:37:44.870 DEBUG:teuthology.orchestra.run.smithi070:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-22T06:37:44.941 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi070.front.sepia.ceph.com 2024-04-22T06:37:44.941 DEBUG:teuthology.orchestra.run.smithi070:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-22T06:37:44.999 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi070.front.sepia.ceph.com 2024-04-22T06:37:45.000 DEBUG:teuthology.orchestra.run.smithi070:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-22T06:37:45.049 INFO:tasks.ceph:Archiving mon data... 2024-04-22T06:37:45.050 DEBUG:teuthology.misc:Transferring archived files from smithi044:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668133/data/mon.a.tgz 2024-04-22T06:37:45.050 DEBUG:teuthology.orchestra.run.smithi044:> mktemp 2024-04-22T06:37:45.067 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/tmp.NdtpSpG5gD 2024-04-22T06:37:45.068 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.NdtpSpG5gD 2024-04-22T06:37:45.197 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 0666 /tmp/tmp.NdtpSpG5gD 2024-04-22T06:37:45.281 DEBUG:teuthology.orchestra.remote:smithi044:/tmp/tmp.NdtpSpG5gD is 273KB 2024-04-22T06:37:45.377 DEBUG:teuthology.orchestra.run.smithi044:> rm -fr /tmp/tmp.NdtpSpG5gD 2024-04-22T06:37:45.392 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668133/data/mon.b.tgz 2024-04-22T06:37:45.392 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2024-04-22T06:37:45.410 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.XEbivin5R9 2024-04-22T06:37:45.410 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.XEbivin5R9 2024-04-22T06:37:45.557 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.XEbivin5R9 2024-04-22T06:37:45.640 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.XEbivin5R9 is 371KB 2024-04-22T06:37:45.735 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.XEbivin5R9 2024-04-22T06:37:45.750 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668133/data/mon.c.tgz 2024-04-22T06:37:45.750 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2024-04-22T06:37:45.804 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.syxQ3Ljct8 2024-04-22T06:37:45.805 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.syxQ3Ljct8 2024-04-22T06:37:45.939 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.syxQ3Ljct8 2024-04-22T06:37:46.019 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.syxQ3Ljct8 is 371KB 2024-04-22T06:37:46.114 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.syxQ3Ljct8 2024-04-22T06:37:46.129 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-22T06:37:46.129 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-22T06:37:46.131 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-22T06:37:46.216 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-22T06:37:46.247 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-22T06:37:46.247 INFO:tasks.ceph:Archiving crash dumps... 2024-04-22T06:37:46.248 DEBUG:teuthology.misc:Transferring archived files from smithi044:/var/lib/ceph/crash to /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668133/remote/smithi044/crash 2024-04-22T06:37:46.249 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-22T06:37:46.324 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/lib/ceph/crash to /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668133/remote/smithi070/crash 2024-04-22T06:37:46.325 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-22T06:37:46.360 INFO:tasks.ceph:Shutting down logrotate 2024-04-22T06:37:46.360 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T06:37:46.363 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T06:37:46.454 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-22T06:37:46.481 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-22T06:37:46.504 INFO:tasks.ceph:Compressing logs... 2024-04-22T06:37:46.504 DEBUG:teuthology.orchestra.run.smithi044:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-22T06:37:46.525 DEBUG:teuthology.orchestra.run.smithi070:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-22T06:37:46.549 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39886.log 2024-04-22T06:37:46.549 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46060.log 2024-04-22T06:37:46.549 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47871.log 2024-04-22T06:37:46.549 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46060.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39886.log.gz 2024-04-22T06:37:46.549 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44190.log 2024-04-22T06:37:46.549 INFO:teuthology.orchestra.run.smithi044.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46060.log.gz 2024-04-22T06:37:46.550 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42881.log 2024-04-22T06:37:46.550 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47787.log 2024-04-22T06:37:46.550 INFO:teuthology.orchestra.run.smithi044.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47871.log.gz 2024-04-22T06:37:46.570 INFO:teuthology.orchestra.run.smithi044.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44190.log.gz 2024-04-22T06:37:46.570 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43209.log 2024-04-22T06:37:46.570 INFO:teuthology.orchestra.run.smithi044.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42881.log.gz 2024-04-22T06:37:46.570 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41643.log 2024-04-22T06:37:46.570 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43209.log: gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47787.log.gz -5 2024-04-22T06:37:46.571 INFO:teuthology.orchestra.run.smithi044.stderr: --verbose -- /var/log/ceph/ceph-client.admin.40268.log 2024-04-22T06:37:46.571 INFO:teuthology.orchestra.run.smithi044.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43209.log.gz 2024-04-22T06:37:46.571 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48142.log 2024-04-22T06:37:46.571 INFO:teuthology.orchestra.run.smithi044.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41643.log.gz 2024-04-22T06:37:46.571 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44064.log 2024-04-22T06:37:46.571 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47138.log 2024-04-22T06:37:46.571 INFO:teuthology.orchestra.run.smithi044.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.40268.log.gz 2024-04-22T06:37:46.571 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40608.log 2024-04-22T06:37:46.571 INFO:teuthology.orchestra.run.smithi044.stderr: 88.4%/var/log/ceph/ceph-client.admin.47138.log: -- replaced with /var/log/ceph/ceph-client.admin.44064.log.gz 2024-04-22T06:37:46.571 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40182.log 2024-04-22T06:37:46.571 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40608.log: gzip -5 88.4% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.47138.log.gz /var/log/ceph/ceph-client.admin.44568.log 2024-04-22T06:37:46.571 INFO:teuthology.orchestra.run.smithi044.stderr: 2024-04-22T06:37:46.571 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40182.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40608.log.gz 2024-04-22T06:37:46.571 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46680.log 2024-04-22T06:37:46.572 INFO:teuthology.orchestra.run.smithi044.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40182.log.gz 2024-04-22T06:37:46.572 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44568.log: gzip -5 --verbose 95.6% -- -- replaced with /var/log/ceph/ceph-client.admin.48142.log.gz /var/log/ceph/ceph-client.admin.45892.log 2024-04-22T06:37:46.572 INFO:teuthology.orchestra.run.smithi044.stderr: 2024-04-22T06:37:46.572 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46680.log: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.44568.log.gz 2024-04-22T06:37:46.572 INFO:teuthology.orchestra.run.smithi044.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47603.log 2024-04-22T06:37:46.572 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47393.log 2024-04-22T06:37:46.572 INFO:teuthology.orchestra.run.smithi044.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45892.log.gz 2024-04-22T06:37:46.572 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47603.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.46680.log.gz 2024-04-22T06:37:46.572 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44274.log 2024-04-22T06:37:46.572 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47393.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47603.log.gz 2024-04-22T06:37:46.572 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40796.log 2024-04-22T06:37:46.572 INFO:teuthology.orchestra.run.smithi044.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47393.log.gz 2024-04-22T06:37:46.572 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40224.log 2024-04-22T06:37:46.572 INFO:teuthology.orchestra.run.smithi044.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44274.log.gz 2024-04-22T06:37:46.573 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40796.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-22T06:37:46.573 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42325.log 2024-04-22T06:37:46.573 INFO:teuthology.orchestra.run.smithi044.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40796.log.gz 2024-04-22T06:37:46.573 INFO:teuthology.orchestra.run.smithi044.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40224.log.gz/var/log/ceph/ceph.log: 2024-04-22T06:37:46.573 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42874.log 2024-04-22T06:37:46.573 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45132.log 2024-04-22T06:37:46.573 INFO:teuthology.orchestra.run.smithi044.stderr: 84.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-22T06:37:46.573 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42874.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42325.log.gz 2024-04-22T06:37:46.573 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42661.log 2024-04-22T06:37:46.573 INFO:teuthology.orchestra.run.smithi044.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42874.log.gz/var/log/ceph/ceph-client.admin.45132.log: 2024-04-22T06:37:46.573 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-22T06:37:46.573 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42661.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45132.log.gz 2024-04-22T06:37:46.573 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42367.log 2024-04-22T06:37:46.573 INFO:teuthology.orchestra.run.smithi044.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42661.log.gz 2024-04-22T06:37:46.573 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47477.log 2024-04-22T06:37:46.574 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42367.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27175.log 2024-04-22T06:37:46.574 INFO:teuthology.orchestra.run.smithi044.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42367.log.gz 2024-04-22T06:37:46.574 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47477.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-22T06:37:46.574 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47477.log.gz 2024-04-22T06:37:46.574 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph.tmp-client.admin.27175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44526.log 2024-04-22T06:37:46.574 INFO:teuthology.orchestra.run.smithi044.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27175.log.gz 2024-04-22T06:37:46.574 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40082.log 2024-04-22T06:37:46.574 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43582.log 2024-04-22T06:37:46.574 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41685.log 2024-04-22T06:37:46.574 INFO:teuthology.orchestra.run.smithi044.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44526.log.gz 2024-04-22T06:37:46.574 INFO:teuthology.orchestra.run.smithi044.stderr: 89.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-22T06:37:46.574 INFO:teuthology.orchestra.run.smithi044.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40082.log.gz 2024-04-22T06:37:46.574 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46879.log 2024-04-22T06:37:46.574 INFO:teuthology.orchestra.run.smithi044.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43582.log.gz 2024-04-22T06:37:46.575 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43581.log 2024-04-22T06:37:46.575 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41685.log: /var/log/ceph/ceph-client.admin.46879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42883.log 2024-04-22T06:37:46.575 INFO:teuthology.orchestra.run.smithi044.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41685.log.gz 2024-04-22T06:37:46.575 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41953.log 2024-04-22T06:37:46.575 INFO:teuthology.orchestra.run.smithi044.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.46879.log.gz 2024-04-22T06:37:46.575 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43581.log.gz 2024-04-22T06:37:46.575 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5/var/log/ceph/ceph-client.admin.42883.log: --verbose -- /var/log/ceph/ceph-client.admin.42451.log 2024-04-22T06:37:46.575 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46813.log 2024-04-22T06:37:46.575 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41953.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42883.log.gz 2024-04-22T06:37:46.575 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40670.log 2024-04-22T06:37:46.575 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42451.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41953.log.gz 2024-04-22T06:37:46.575 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47290.log 2024-04-22T06:37:46.575 INFO:teuthology.orchestra.run.smithi044.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42451.log.gz 2024-04-22T06:37:46.575 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44442.log 2024-04-22T06:37:46.575 INFO:teuthology.orchestra.run.smithi044.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40670.log.gz 2024-04-22T06:37:46.576 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47290.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.46813.log.gz 2024-04-22T06:37:46.576 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47519.log 2024-04-22T06:37:46.576 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-22T06:37:46.576 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44442.log: /var/log/ceph/ceph-client.admin.47519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44400.log 2024-04-22T06:37:46.576 INFO:teuthology.orchestra.run.smithi044.stderr: 90.7% 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44442.log.gz -- replaced with /var/log/ceph/ceph-client.admin.47290.log.gz 2024-04-22T06:37:46.576 INFO:teuthology.orchestra.run.smithi044.stderr: 2024-04-22T06:37:46.576 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47519.log.gz 2024-04-22T06:37:46.576 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46945.log 2024-04-22T06:37:46.576 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46456.log 2024-04-22T06:37:46.576 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44400.log: /var/log/ceph/ceph-client.admin.46945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45766.log 2024-04-22T06:37:46.576 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44400.log.gz 2024-04-22T06:37:46.576 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42037.log 2024-04-22T06:37:46.576 INFO:teuthology.orchestra.run.smithi044.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.46945.log.gz 2024-04-22T06:37:46.576 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44106.log 2024-04-22T06:37:46.577 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46456.log.gz 2024-04-22T06:37:46.577 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42037.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45766.log.gz 2024-04-22T06:37:46.577 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40880.log 2024-04-22T06:37:46.577 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46102.log 2024-04-22T06:37:46.577 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42037.log.gz 2024-04-22T06:37:46.577 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44106.log: /var/log/ceph/ceph-client.admin.40880.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44106.log.gz 2024-04-22T06:37:46.577 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47561.log 2024-04-22T06:37:46.577 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44698.log 2024-04-22T06:37:46.577 INFO:teuthology.orchestra.run.smithi044.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40880.log.gz 2024-04-22T06:37:46.577 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46102.log: /var/log/ceph/ceph-client.admin.47561.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46102.log.gz 2024-04-22T06:37:46.577 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47703.log 2024-04-22T06:37:46.577 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-22T06:37:46.577 INFO:teuthology.orchestra.run.smithi044.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47561.log.gz 2024-04-22T06:37:46.577 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44698.log: /var/log/ceph/ceph-client.admin.47703.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44698.log.gz 2024-04-22T06:37:46.578 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33469.log 2024-04-22T06:37:46.578 INFO:teuthology.orchestra.run.smithi044.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47703.log.gz 2024-04-22T06:37:46.578 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42199.log 2024-04-22T06:37:46.578 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-mds.c.log: /var/log/ceph/ceph-client.admin.33469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42884.log 2024-04-22T06:37:46.578 INFO:teuthology.orchestra.run.smithi044.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33469.log.gz 2024-04-22T06:37:46.578 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42199.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-22T06:37:46.578 INFO:teuthology.orchestra.run.smithi044.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42199.log.gz 2024-04-22T06:37:46.578 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40394.log 2024-04-22T06:37:46.578 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42884.log: /var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43578.log 2024-04-22T06:37:46.578 INFO:teuthology.orchestra.run.smithi044.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42884.log.gz 2024-04-22T06:37:46.578 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40394.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-22T06:37:46.578 INFO:teuthology.orchestra.run.smithi044.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40394.log.gz 2024-04-22T06:37:46.578 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42882.log 2024-04-22T06:37:46.578 INFO:teuthology.orchestra.run.smithi044.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43578.log.gz 2024-04-22T06:37:46.579 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-22T06:37:46.579 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40310.log 2024-04-22T06:37:46.579 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4%/var/log/ceph/ceph-osd.0.log: -- replaced with /var/log/ceph/ceph-client.admin.42882.log.gz 2024-04-22T06:37:46.579 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41030.log 2024-04-22T06:37:46.579 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44484.log 2024-04-22T06:37:46.579 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40310.log: /var/log/ceph/ceph-client.admin.41030.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40310.log.gz 2024-04-22T06:37:46.579 INFO:teuthology.orchestra.run.smithi044.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41030.log.gz 2024-04-22T06:37:46.579 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47011.log 2024-04-22T06:37:46.594 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33344.log 2024-04-22T06:37:46.594 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33304.log 2024-04-22T06:37:46.594 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-22T06:37:46.594 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.33344.log: /var/log/ceph/ceph-client.admin.33304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33515.log 2024-04-22T06:37:46.594 INFO:teuthology.orchestra.run.smithi070.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33344.log.gz 2024-04-22T06:37:46.594 INFO:teuthology.orchestra.run.smithi070.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33304.log.gz 2024-04-22T06:37:46.594 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.f.log 2024-04-22T06:37:46.594 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.33515.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-22T06:37:46.594 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-mds.f.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.33515.log.gz 2024-04-22T06:37:46.595 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-22T06:37:46.595 INFO:teuthology.orchestra.run.smithi070.stderr: 91.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-22T06:37:46.595 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-22T06:37:46.595 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-22T06:37:46.595 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33555.log 2024-04-22T06:37:46.595 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-22T06:37:46.595 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.33555.log: gzip 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-22T06:37:46.595 INFO:teuthology.orchestra.run.smithi070.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.33595.log 2024-04-22T06:37:46.595 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-osd.4.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33555.log.gz 2024-04-22T06:37:46.595 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-22T06:37:46.595 INFO:teuthology.orchestra.run.smithi070.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mds.f.log.gz 2024-04-22T06:37:46.595 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.33595.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-22T06:37:46.595 INFO:teuthology.orchestra.run.smithi070.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33595.log.gz 2024-04-22T06:37:46.595 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-22T06:37:46.596 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33384.log 2024-04-22T06:37:46.596 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33475.log 2024-04-22T06:37:46.596 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.33384.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.33384.log.gz 2024-04-22T06:37:46.596 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-22T06:37:46.596 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.33475.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33475.log.gz 2024-04-22T06:37:46.596 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-22T06:37:46.596 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-mds.d.log: 90.7% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-22T06:37:46.596 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33428.log 2024-04-22T06:37:46.608 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-mon.b.log: 92.6% -- replaced with /var/log/ceph/ceph-mds.d.log.gz/var/log/ceph/ceph-client.admin.33428.log: 2024-04-22T06:37:46.608 INFO:teuthology.orchestra.run.smithi070.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.33428.log.gz 2024-04-22T06:37:46.619 INFO:teuthology.orchestra.run.smithi070.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-22T06:37:46.634 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44484.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44484.log.gz 2024-04-22T06:37:46.634 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43220.log 2024-04-22T06:37:46.634 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47011.log: 92.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46414.log 2024-04-22T06:37:46.634 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-22T06:37:46.634 INFO:teuthology.orchestra.run.smithi044.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47011.log.gz 2024-04-22T06:37:46.634 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44316.log 2024-04-22T06:37:46.634 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43220.log: /var/log/ceph/ceph-client.admin.46414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43576.log 2024-04-22T06:37:46.634 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44316.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43220.log.gz 2024-04-22T06:37:46.634 INFO:teuthology.orchestra.run.smithi044.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46414.log.gz 2024-04-22T06:37:46.634 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44316.log.gz 2024-04-22T06:37:46.634 INFO:teuthology.orchestra.run.smithi044.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43213.log 2024-04-22T06:37:46.635 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43576.log: gzip -5 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43576.log.gz 2024-04-22T06:37:46.635 INFO:teuthology.orchestra.run.smithi044.stderr: --verbose -- /var/log/ceph/ceph-client.admin.33202.log 2024-04-22T06:37:46.635 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42121.log 88.8% 2024-04-22T06:37:46.635 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43213.log.gz 2024-04-22T06:37:46.635 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.33202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48322.log 2024-04-22T06:37:46.635 INFO:teuthology.orchestra.run.smithi044.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33202.log.gz/var/log/ceph/ceph-client.admin.42121.log: 2024-04-22T06:37:46.635 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41769.log 2024-04-22T06:37:46.635 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42121.log.gz 2024-04-22T06:37:46.635 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41460.log 2024-04-22T06:37:46.635 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48322.log: /var/log/ceph/ceph-client.admin.41769.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-22T06:37:46.635 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41460.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41769.log.gz 2024-04-22T06:37:46.635 INFO:teuthology.orchestra.run.smithi044.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.41460.log.gz 2024-04-22T06:37:46.635 INFO:teuthology.orchestra.run.smithi044.stderr: 95.7% -- replaced with /var/log/ceph/ceph-client.admin.48322.log.gz 2024-04-22T06:37:46.635 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44614.log 2024-04-22T06:37:46.636 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40436.log 2024-04-22T06:37:46.636 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44941.log 2024-04-22T06:37:46.636 INFO:teuthology.orchestra.run.smithi044.stderr: 90.5% 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44614.log.gz 2024-04-22T06:37:46.636 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-22T06:37:46.636 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40436.log: gzip 88.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.43212.log 2024-04-22T06:37:46.636 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40436.log.gz 2024-04-22T06:37:46.636 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46498.log 2024-04-22T06:37:46.636 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44010.log 2024-04-22T06:37:46.636 INFO:teuthology.orchestra.run.smithi044.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.44941.log.gz 2024-04-22T06:37:46.636 INFO:teuthology.orchestra.run.smithi044.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43212.log.gz 2024-04-22T06:37:46.636 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47247.log 2024-04-22T06:37:46.636 INFO:teuthology.orchestra.run.smithi044.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46498.log.gz 2024-04-22T06:37:46.636 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42409.log 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44010.log.gz 2024-04-22T06:37:46.637 INFO:teuthology.orchestra.run.smithi044.stderr: 2024-04-22T06:37:46.637 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45850.log 2024-04-22T06:37:46.637 INFO:teuthology.orchestra.run.smithi044.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47247.log.gz 2024-04-22T06:37:46.637 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43779.log 2024-04-22T06:37:46.637 INFO:teuthology.orchestra.run.smithi044.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42409.log.gz/var/log/ceph/ceph-client.admin.45850.log: 2024-04-22T06:37:46.637 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42577.log 2024-04-22T06:37:46.637 INFO:teuthology.orchestra.run.smithi044.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45850.log.gz 2024-04-22T06:37:46.637 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40964.log 2024-04-22T06:37:46.637 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43779.log: /var/log/ceph/ceph-client.admin.42577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44809.log 2024-04-22T06:37:46.637 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40964.log: 88.4% 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43779.log.gz 2024-04-22T06:37:46.637 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42577.log.gz 2024-04-22T06:37:46.637 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43580.log 2024-04-22T06:37:46.637 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45808.log 2024-04-22T06:37:46.637 INFO:teuthology.orchestra.run.smithi044.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.40964.log.gz 2024-04-22T06:37:46.638 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44809.log: /var/log/ceph/ceph-client.admin.43580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40922.log 2024-04-22T06:37:46.638 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43580.log.gz/var/log/ceph/ceph-client.admin.45808.log: 2024-04-22T06:37:46.638 INFO:teuthology.orchestra.run.smithi044.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.44809.log.gz 2024-04-22T06:37:46.638 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45808.log.gz 2024-04-22T06:37:46.638 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41911.log 2024-04-22T06:37:46.638 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42877.log 2024-04-22T06:37:46.638 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40922.log: /var/log/ceph/ceph-client.admin.41911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45526.log 2024-04-22T06:37:46.638 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42877.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40922.log.gz 2024-04-22T06:37:46.638 INFO:teuthology.orchestra.run.smithi044.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41911.log.gz 2024-04-22T06:37:46.638 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43863.log 2024-04-22T06:37:46.638 INFO:teuthology.orchestra.run.smithi044.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42877.log.gz 2024-04-22T06:37:46.638 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41995.log 2024-04-22T06:37:46.638 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.45526.log: /var/log/ceph/ceph-client.admin.43863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40352.log 2024-04-22T06:37:46.638 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41995.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43863.log.gz 2024-04-22T06:37:46.639 INFO:teuthology.orchestra.run.smithi044.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41995.log.gz 2024-04-22T06:37:46.639 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40754.log 2024-04-22T06:37:46.639 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41727.log 2024-04-22T06:37:46.639 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40754.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40352.log.gz 2024-04-22T06:37:46.639 INFO:teuthology.orchestra.run.smithi044.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40754.log.gz 2024-04-22T06:37:46.639 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43905.log 2024-04-22T06:37:46.639 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41727.log: 87.5% 93.8% -- replaced with /var/log/ceph/ceph-client.admin.41727.log.gz 2024-04-22T06:37:46.639 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.0.45526.log.gz 2024-04-22T06:37:46.639 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47661.log 2024-04-22T06:37:46.639 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40480.log 2024-04-22T06:37:46.639 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47661.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43905.log.gz 2024-04-22T06:37:46.639 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40712.log 2024-04-22T06:37:46.639 INFO:teuthology.orchestra.run.smithi044.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47661.log.gz 2024-04-22T06:37:46.639 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.e.log 2024-04-22T06:37:46.640 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40480.log: /var/log/ceph/ceph-client.admin.40712.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40480.log.gz 2024-04-22T06:37:46.640 INFO:teuthology.orchestra.run.smithi044.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40712.log.gz 2024-04-22T06:37:46.640 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40524.log 2024-04-22T06:37:46.640 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46228.log 2024-04-22T06:37:46.640 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-mds.e.log: /var/log/ceph/ceph-client.admin.40524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42493.log 2024-04-22T06:37:46.640 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46228.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40524.log.gz 2024-04-22T06:37:46.640 INFO:teuthology.orchestra.run.smithi044.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46228.log.gz 2024-04-22T06:37:46.640 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42872.log 2024-04-22T06:37:46.640 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40566.log 2024-04-22T06:37:46.640 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42872.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42493.log.gz 2024-04-22T06:37:46.640 INFO:teuthology.orchestra.run.smithi044.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42872.log.gz 2024-04-22T06:37:46.640 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42703.log 2024-04-22T06:37:46.640 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40566.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40566.log.gz 2024-04-22T06:37:46.640 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44358.log 2024-04-22T06:37:46.641 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43215.log 2024-04-22T06:37:46.641 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44358.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42703.log.gz 2024-04-22T06:37:46.641 INFO:teuthology.orchestra.run.smithi044.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44358.log.gz 2024-04-22T06:37:46.641 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42619.log 2024-04-22T06:37:46.641 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43215.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43215.log.gz 2024-04-22T06:37:46.641 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41811.log 2024-04-22T06:37:46.641 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42885.log 2024-04-22T06:37:46.641 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41811.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42619.log.gz 2024-04-22T06:37:46.641 INFO:teuthology.orchestra.run.smithi044.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41811.log.gz 2024-04-22T06:37:46.641 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39967.log 2024-04-22T06:37:46.641 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42885.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42885.log.gz 2024-04-22T06:37:46.641 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46330.log 2024-04-22T06:37:46.641 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46288.log 2024-04-22T06:37:46.641 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46330.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39967.log.gz 2024-04-22T06:37:46.641 INFO:teuthology.orchestra.run.smithi044.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46330.log.gz 2024-04-22T06:37:46.642 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46746.log 2024-04-22T06:37:46.642 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46288.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46288.log.gz 2024-04-22T06:37:46.642 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44740.log 2024-04-22T06:37:46.642 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44148.log 2024-04-22T06:37:46.642 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44740.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.46746.log.gz 2024-04-22T06:37:46.642 INFO:teuthology.orchestra.run.smithi044.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.44740.log.gz 2024-04-22T06:37:46.642 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46598.log 2024-04-22T06:37:46.642 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44148.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44148.log.gz 2024-04-22T06:37:46.642 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43218.log 2024-04-22T06:37:46.642 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45934.log 2024-04-22T06:37:46.642 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43218.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46598.log.gz 2024-04-22T06:37:46.642 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43218.log.gz 2024-04-22T06:37:46.642 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44875.log 2024-04-22T06:37:46.642 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45934.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45934.log.gz 2024-04-22T06:37:46.643 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46540.log 2024-04-22T06:37:46.643 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42283.log 2024-04-22T06:37:46.643 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46540.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46540.log.gz 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44875.log.gz 2024-04-22T06:37:46.643 INFO:teuthology.orchestra.run.smithi044.stderr: 2024-04-22T06:37:46.643 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44232.log 2024-04-22T06:37:46.643 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42241.log 2024-04-22T06:37:46.643 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44232.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42283.log.gz 2024-04-22T06:37:46.643 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44232.log.gz 2024-04-22T06:37:46.643 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46144.log 2024-04-22T06:37:46.643 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42241.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42241.log.gz 2024-04-22T06:37:46.643 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43821.log 2024-04-22T06:37:46.643 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43583.log 2024-04-22T06:37:46.643 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43821.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46144.log.gz 2024-04-22T06:37:46.643 INFO:teuthology.orchestra.run.smithi044.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43821.log.gz 2024-04-22T06:37:46.644 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45976.log 2024-04-22T06:37:46.644 INFO:teuthology.orchestra.run.smithi044.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mds.e.log.gz 2024-04-22T06:37:46.644 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47745.log 2024-04-22T06:37:46.644 INFO:teuthology.orchestra.run.smithi044.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43583.log.gz 2024-04-22T06:37:46.644 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43956.log 2024-04-22T06:37:46.644 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45976.log: /var/log/ceph/ceph-client.admin.47745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43571.log 2024-04-22T06:37:46.644 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43956.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45976.log.gz 2024-04-22T06:37:46.644 INFO:teuthology.orchestra.run.smithi044.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47745.log.gz 2024-04-22T06:37:46.644 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-22T06:37:46.644 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43956.log.gz 2024-04-22T06:37:46.644 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46018.log 2024-04-22T06:37:46.644 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43571.log: gzip/var/log/ceph/ceph-mds.a.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.47435.log 2024-04-22T06:37:46.644 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46018.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43571.log.gz 2024-04-22T06:37:46.644 INFO:teuthology.orchestra.run.smithi044.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46018.log.gz 2024-04-22T06:37:46.645 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43443.log 2024-04-22T06:37:46.645 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44656.log 2024-04-22T06:37:46.645 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47435.log: /var/log/ceph/ceph-client.admin.43443.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47435.log.gz 2024-04-22T06:37:46.645 INFO:teuthology.orchestra.run.smithi044.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43443.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45724.log 2024-04-22T06:37:46.645 INFO:teuthology.orchestra.run.smithi044.stderr: 2024-04-22T06:37:46.645 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44656.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44656.log.gz 2024-04-22T06:37:46.645 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42079.log 2024-04-22T06:37:46.645 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33580.log 2024-04-22T06:37:46.645 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45724.log: /var/log/ceph/ceph-client.admin.42079.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45724.log.gz 2024-04-22T06:37:46.645 INFO:teuthology.orchestra.run.smithi044.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42079.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43214.log 2024-04-22T06:37:46.645 INFO:teuthology.orchestra.run.smithi044.stderr: 2024-04-22T06:37:46.645 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.33580.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33580.log.gz 2024-04-22T06:37:46.645 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47181.log 2024-04-22T06:37:46.645 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43221.log 2024-04-22T06:37:46.646 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43214.log: /var/log/ceph/ceph-client.admin.47181.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43214.log.gz 2024-04-22T06:37:46.646 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42535.log 2024-04-22T06:37:46.646 INFO:teuthology.orchestra.run.smithi044.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.47181.log.gz 2024-04-22T06:37:46.646 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43221.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43221.log.gz 2024-04-22T06:37:46.646 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46186.log 2024-04-22T06:37:46.646 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41853.log 2024-04-22T06:37:46.646 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42535.log: /var/log/ceph/ceph-client.admin.46186.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42535.log.gz 2024-04-22T06:37:46.646 INFO:teuthology.orchestra.run.smithi044.stderr: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45007.log 2024-04-22T06:37:46.646 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46186.log.gz 2024-04-22T06:37:46.646 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41853.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41853.log.gz 2024-04-22T06:37:46.646 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47054.log 2024-04-22T06:37:46.646 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40838.log 2024-04-22T06:37:46.646 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45007.log: /var/log/ceph/ceph-client.admin.47054.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.45007.log.gz 2024-04-22T06:37:46.646 INFO:teuthology.orchestra.run.smithi044.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.47054.log.gz 2024-04-22T06:37:46.647 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45682.log 2024-04-22T06:37:46.647 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40838.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40838.log.gz 2024-04-22T06:37:46.647 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46372.log 2024-04-22T06:37:46.647 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47829.log 2024-04-22T06:37:46.647 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45682.log: /var/log/ceph/ceph-client.admin.46372.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45682.log.gz 2024-04-22T06:37:46.647 INFO:teuthology.orchestra.run.smithi044.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46372.log.gz 2024-04-22T06:37:46.647 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47829.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47829.log.gz 2024-04-22T06:37:46.647 INFO:teuthology.orchestra.run.smithi044.stderr: 90.1% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-22T06:37:46.648 INFO:teuthology.orchestra.run.smithi070.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-22T06:37:46.692 INFO:teuthology.orchestra.run.smithi070.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-22T06:37:46.745 INFO:teuthology.orchestra.run.smithi044.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-22T06:37:47.136 INFO:teuthology.orchestra.run.smithi044.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-22T06:37:47.169 INFO:teuthology.orchestra.run.smithi070.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-22T06:37:47.177 INFO:teuthology.orchestra.run.smithi070.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-22T06:37:47.189 INFO:teuthology.orchestra.run.smithi044.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-22T06:37:47.198 INFO:teuthology.orchestra.run.smithi070.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-22T06:37:47.211 INFO:teuthology.orchestra.run.smithi070.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-22T06:37:47.213 INFO:teuthology.orchestra.run.smithi070.stderr: 2024-04-22T06:37:47.213 INFO:teuthology.orchestra.run.smithi070.stderr:real 0m0.655s 2024-04-22T06:37:47.213 INFO:teuthology.orchestra.run.smithi070.stderr:user 0m2.649s 2024-04-22T06:37:47.213 INFO:teuthology.orchestra.run.smithi070.stderr:sys 0m0.124s 2024-04-22T06:37:47.236 INFO:teuthology.orchestra.run.smithi044.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-22T06:37:47.253 INFO:teuthology.orchestra.run.smithi044.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-22T06:37:47.255 INFO:teuthology.orchestra.run.smithi044.stderr: 2024-04-22T06:37:47.255 INFO:teuthology.orchestra.run.smithi044.stderr:real 0m0.717s 2024-04-22T06:37:47.255 INFO:teuthology.orchestra.run.smithi044.stderr:user 0m2.757s 2024-04-22T06:37:47.255 INFO:teuthology.orchestra.run.smithi044.stderr:sys 0m0.220s 2024-04-22T06:37:47.256 INFO:tasks.ceph:Archiving logs... 2024-04-22T06:37:47.256 DEBUG:teuthology.misc:Transferring archived files from smithi044:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668133/remote/smithi044/log 2024-04-22T06:37:47.257 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-22T06:37:47.823 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668133/remote/smithi070/log 2024-04-22T06:37:47.824 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-22T06:37:48.241 ERROR:teuthology.run_tasks:Manager failed: ceph Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1918, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 310, in task stop_all_quiescers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 260, in stop_all_quiescers thrasher.stop() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 252, in stop self.tell_quiesce_leader( "--cancel", "--all" ) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 110, in tell_quiesce_leader leader = self.fs.get_var('qdb_leader') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 959, in get_var return self.get_mds_map(status=status)[var] File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 955, in get_mds_map status = self.status() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 408, in status return FSStatus(self.mon_manager, epoch) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 88, in __init__ self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout return self.run_ceph_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd return self.mon_manager.run_cluster_cmd(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2822, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2654, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2420, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1656, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2024-04-22T06:37:48.241 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-22T06:37:48.251 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 310, in task stop_all_quiescers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 260, in stop_all_quiescers thrasher.stop() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 252, in stop self.tell_quiesce_leader( "--cancel", "--all" ) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 110, in tell_quiesce_leader leader = self.fs.get_var('qdb_leader') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 959, in get_var return self.get_mds_map(status=status)[var] File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 955, in get_mds_map status = self.status() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 408, in status return FSStatus(self.mon_manager, epoch) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 88, in __init__ self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout return self.run_ceph_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd return self.mon_manager.run_cluster_cmd(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' 2024-04-22T06:37:48.251 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-22T06:37:48.252 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-22T06:37:48.254 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-22T06:37:48.346 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-22T06:37:48.355 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-22T06:37:48.356 DEBUG:teuthology.orchestra.run.smithi044:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T06:37:48.359 DEBUG:teuthology.orchestra.run.smithi070:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T06:37:48.374 INFO:teuthology.orchestra.run.smithi044.stderr:bash: line 1: ntpq: command not found 2024-04-22T06:37:48.374 INFO:teuthology.orchestra.run.smithi070.stderr:bash: line 1: ntpq: command not found 2024-04-22T06:37:48.378 INFO:teuthology.orchestra.run.smithi044.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-22T06:37:48.378 INFO:teuthology.orchestra.run.smithi044.stdout:=============================================================================== 2024-04-22T06:37:48.378 INFO:teuthology.orchestra.run.smithi044.stdout:^- hv01.front.sepia.ceph.com 2 6 377 59 +1165us[+1161us] +/- 581ms 2024-04-22T06:37:48.378 INFO:teuthology.orchestra.run.smithi044.stdout:^* hv02.front.sepia.ceph.com 3 6 377 58 -151us[ -154us] +/- 61ms 2024-04-22T06:37:48.378 INFO:teuthology.orchestra.run.smithi044.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 58 +155us[ +155us] +/- 68ms 2024-04-22T06:37:48.378 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T06:37:48.380 INFO:teuthology.orchestra.run.smithi070.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-22T06:37:48.380 INFO:teuthology.orchestra.run.smithi070.stdout:=============================================================================== 2024-04-22T06:37:48.380 INFO:teuthology.orchestra.run.smithi070.stdout:^- hv01.front.sepia.ceph.com 2 6 377 57 +1162us[+1162us] +/- 581ms 2024-04-22T06:37:48.380 INFO:teuthology.orchestra.run.smithi070.stdout:^* hv02.front.sepia.ceph.com 3 6 377 59 -135us[ -131us] +/- 61ms 2024-04-22T06:37:48.380 INFO:teuthology.orchestra.run.smithi070.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 60 +169us[ +173us] +/- 68ms 2024-04-22T06:37:48.380 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T06:37:48.381 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-22T06:37:48.438 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-22T06:37:48.438 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-22T06:37:48.447 DEBUG:teuthology.orchestra.run.smithi044:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-22T06:37:48.519 DEBUG:teuthology.orchestra.run.smithi070:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-22T06:37:48.583 DEBUG:teuthology.orchestra.run.smithi044:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-22T06:37:48.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T06:37:48.613 DEBUG:teuthology.orchestra.run.smithi070:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-22T06:37:48.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T06:37:48.641 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-22T06:37:48.650 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-22T06:37:48.660 INFO:teuthology.task.internal:Duration was 793.315355 seconds 2024-04-22T06:37:48.660 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-22T06:37:48.670 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-22T06:37:48.670 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-22T06:37:48.673 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-22T06:37:48.711 INFO:teuthology.orchestra.run.smithi044.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-22T06:37:48.720 INFO:teuthology.orchestra.run.smithi070.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-22T06:37:49.088 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-22T06:37:49.088 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi044.front.sepia.ceph.com 2024-04-22T06:37:49.088 DEBUG:teuthology.orchestra.run.smithi044:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-22T06:37:49.150 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi070.front.sepia.ceph.com 2024-04-22T06:37:49.150 DEBUG:teuthology.orchestra.run.smithi070:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-22T06:37:49.180 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-22T06:37:49.180 DEBUG:teuthology.orchestra.run.smithi044:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-22T06:37:49.193 DEBUG:teuthology.orchestra.run.smithi070:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-22T06:37:49.248 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-22T06:37:49.248 DEBUG:teuthology.orchestra.run.smithi044:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-22T06:37:49.261 DEBUG:teuthology.orchestra.run.smithi070:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-22T06:37:49.385 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-22T06:37:49.396 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-22T06:37:49.396 DEBUG:teuthology.orchestra.run.smithi044:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-22T06:37:49.421 DEBUG:teuthology.orchestra.run.smithi070:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-22T06:37:49.444 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-22T06:37:49.456 DEBUG:teuthology.orchestra.run.smithi044:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-22T06:37:49.464 DEBUG:teuthology.orchestra.run.smithi070:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-22T06:37:49.484 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern = core 2024-04-22T06:37:49.507 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern = core 2024-04-22T06:37:49.529 DEBUG:teuthology.orchestra.run.smithi044:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-22T06:37:49.560 WARNING:teuthology.task.internal:Found coredumps on ubuntu@smithi044.front.sepia.ceph.com, flagging run as failed 2024-04-22T06:37:49.561 DEBUG:teuthology.orchestra.run.smithi070:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-22T06:37:49.584 WARNING:teuthology.task.internal:Found coredumps on ubuntu@smithi070.front.sepia.ceph.com, flagging run as failed 2024-04-22T06:37:49.584 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-22T06:37:49.597 INFO:teuthology.task.internal:Transferring archived files... 2024-04-22T06:37:49.598 DEBUG:teuthology.misc:Transferring archived files from smithi044:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668133/remote/smithi044 2024-04-22T06:37:49.598 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-22T06:38:13.904 INFO:teuthology.task.internal:Transferring binaries for coredumps... 2024-04-22T06:38:13.915 INFO:teuthology.task.internal: core looks like: /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668133/remote/smithi044/coredump/1713767575.40084.core.gz: gzip compressed data, was "1713767575.40084.core", last modified: Mon Apr 22 06:38:05 2024, max compression, original size modulo 2^32 261677056 2024-04-22T06:38:13.915 INFO:teuthology.task.internal:core is compressed, try accessing gzip file ... 2024-04-22T06:38:15.088 INFO:teuthology.task.internal: core looks like: /tmp/tmpm4xtmq8l: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i c', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2024-04-22T06:38:15.121 INFO:teuthology.task.internal: dump_program: ceph-mds -f --cluster ceph -i c 2024-04-22T06:38:15.121 DEBUG:teuthology.orchestra.run.smithi044:> which 'ceph-mds -f --cluster ceph -i c' 2024-04-22T06:38:15.152 INFO:teuthology.orchestra.run.smithi044.stderr:/usr/bin/which: invalid option -- 'f' 2024-04-22T06:38:15.152 INFO:teuthology.orchestra.run.smithi044.stderr:/usr/bin/which: unrecognized option '--cluster' 2024-04-22T06:38:15.152 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph-mds 2024-04-22T06:38:15.152 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph 2024-04-22T06:38:15.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T06:38:15.153 INFO:teuthology.orchestra.run.smithi044.stderr:/usr/bin/which: no c in (/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin) 2024-04-22T06:38:15.154 ERROR:teuthology.run_tasks:Manager failed: internal.archive Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 392, in archive yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 310, in task stop_all_quiescers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 260, in stop_all_quiescers thrasher.stop() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 252, in stop self.tell_quiesce_leader( "--cancel", "--all" ) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 110, in tell_quiesce_leader leader = self.fs.get_var('qdb_leader') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 959, in get_var return self.get_mds_map(status=status)[var] File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 955, in get_mds_map status = self.status() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 408, in status return FSStatus(self.mon_manager, epoch) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 88, in __init__ self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout return self.run_ceph_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd return self.mon_manager.run_cluster_cmd(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 420, in archive fetch_binaries_for_coredumps(path, rem) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 345, in fetch_binaries_for_coredumps remote_path = remote.sh(['which', dump_program]).rstrip() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 1: "which 'ceph-mds -f --cluster ceph -i c'" 2024-04-22T06:38:15.155 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-22T06:38:15.166 INFO:teuthology.task.internal:Not uploading archives. 2024-04-22T06:38:15.166 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-22T06:38:15.176 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-22T06:38:15.176 DEBUG:teuthology.orchestra.run.smithi044:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-22T06:38:15.197 DEBUG:teuthology.orchestra.run.smithi070:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-22T06:38:15.212 INFO:teuthology.orchestra.run.smithi044.stdout: 262213 4 drwxr-xr-x 4 ubuntu ubuntu 4096 Apr 22 06:37 /home/ubuntu/cephtest 2024-04-22T06:38:15.213 INFO:teuthology.orchestra.run.smithi044.stdout: 410661 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 22 06:33 /home/ubuntu/cephtest/mnt.0 2024-04-22T06:38:15.213 INFO:teuthology.orchestra.run.smithi044.stdout: 262214 4 drwxr-xr-x 5 ubuntu ubuntu 4096 Apr 22 06:37 /home/ubuntu/cephtest/archive 2024-04-22T06:38:15.213 INFO:teuthology.orchestra.run.smithi044.stdout: 395989 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 22 06:37 /home/ubuntu/cephtest/archive/audit 2024-04-22T06:38:15.213 INFO:teuthology.orchestra.run.smithi044.stdout: 396272 136 -rw------- 1 ubuntu ubuntu 136801 Apr 22 06:37 /home/ubuntu/cephtest/archive/audit/audit.log.gz 2024-04-22T06:38:15.213 INFO:teuthology.orchestra.run.smithi044.stdout: 262216 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 22 06:37 /home/ubuntu/cephtest/archive/syslog 2024-04-22T06:38:15.213 INFO:teuthology.orchestra.run.smithi044.stdout: 262259 4 -rw-rw-rw- 1 ubuntu ubuntu 29 Apr 22 06:24 /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2024-04-22T06:38:15.213 INFO:teuthology.orchestra.run.smithi044.stdout: 262217 188 -rw-r--r-- 1 ubuntu ubuntu 189769 Apr 22 06:37 /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-22T06:38:15.213 INFO:teuthology.orchestra.run.smithi044.stdout: 262218 4 -rw-rw-rw- 1 ubuntu ubuntu 29 Apr 22 06:24 /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2024-04-22T06:38:15.213 INFO:teuthology.orchestra.run.smithi044.stdout: 262215 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 22 06:32 /home/ubuntu/cephtest/archive/coredump 2024-04-22T06:38:15.213 INFO:teuthology.orchestra.run.smithi044.stdout: 263990 28996 -rw------- 1 root root 262291456 Apr 22 06:32 /home/ubuntu/cephtest/archive/coredump/1713767575.40083.core 2024-04-22T06:38:15.213 INFO:teuthology.orchestra.run.smithi044.stdout: 263991 29072 -rw------- 1 root root 262209536 Apr 22 06:32 /home/ubuntu/cephtest/archive/coredump/1713767575.40085.core 2024-04-22T06:38:15.213 INFO:teuthology.orchestra.run.smithi044.stdout: 263989 30592 -rw------- 1 root root 261677056 Apr 22 06:32 /home/ubuntu/cephtest/archive/coredump/1713767575.40084.core 2024-04-22T06:38:15.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T06:38:15.214 INFO:teuthology.orchestra.run.smithi044.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-04-22T06:38:15.215 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 310, in task stop_all_quiescers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 260, in stop_all_quiescers thrasher.stop() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 252, in stop self.tell_quiesce_leader( "--cancel", "--all" ) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 110, in tell_quiesce_leader leader = self.fs.get_var('qdb_leader') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 959, in get_var return self.get_mds_map(status=status)[var] File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 955, in get_mds_map status = self.status() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 408, in status return FSStatus(self.mon_manager, epoch) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 88, in __init__ self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout return self.run_ceph_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd return self.mon_manager.run_cluster_cmd(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-04-22T06:38:15.215 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-22T06:38:15.226 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-22T06:38:15.237 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi044 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' 2024-04-22T06:38:15.237 INFO:teuthology.nuke:Checking targets against current locks 2024-04-22T06:38:15.250 INFO:teuthology.orchestra.run.smithi070.stdout: 262213 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Apr 22 06:37 /home/ubuntu/cephtest 2024-04-22T06:38:15.250 INFO:teuthology.orchestra.run.smithi070.stdout: 262214 4 drwxr-xr-x 5 ubuntu ubuntu 4096 Apr 22 06:37 /home/ubuntu/cephtest/archive 2024-04-22T06:38:15.250 INFO:teuthology.orchestra.run.smithi070.stdout: 262259 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 22 06:37 /home/ubuntu/cephtest/archive/audit 2024-04-22T06:38:15.250 INFO:teuthology.orchestra.run.smithi070.stdout: 262268 112 -rw------- 1 ubuntu ubuntu 114071 Apr 22 06:37 /home/ubuntu/cephtest/archive/audit/audit.log.gz 2024-04-22T06:38:15.250 INFO:teuthology.orchestra.run.smithi070.stdout: 262216 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 22 06:37 /home/ubuntu/cephtest/archive/syslog 2024-04-22T06:38:15.250 INFO:teuthology.orchestra.run.smithi070.stdout: 262267 4 -rw-rw-rw- 1 ubuntu ubuntu 29 Apr 22 06:24 /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2024-04-22T06:38:15.250 INFO:teuthology.orchestra.run.smithi070.stdout: 262217 180 -rw-r--r-- 1 ubuntu ubuntu 181491 Apr 22 06:37 /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-22T06:38:15.250 INFO:teuthology.orchestra.run.smithi070.stdout: 262218 4 -rw-rw-rw- 1 ubuntu ubuntu 29 Apr 22 06:24 /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2024-04-22T06:38:15.250 INFO:teuthology.orchestra.run.smithi070.stdout: 262215 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 22 06:32 /home/ubuntu/cephtest/archive/coredump 2024-04-22T06:38:15.250 INFO:teuthology.orchestra.run.smithi070.stdout: 275755 31588 -rw------- 1 root root 262443008 Apr 22 06:32 /home/ubuntu/cephtest/archive/coredump/1713767575.39570.core 2024-04-22T06:38:15.251 INFO:teuthology.orchestra.run.smithi070.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-04-22T06:38:15.275 DEBUG:teuthology.nuke:shortname: smithi044 2024-04-22T06:38:15.275 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-22T06:38:15.311 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi044.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668133', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-22 06:18:48.194634', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOkhDsKQt1CCnIug9tZrc1CemwV6jDC9sr/9o9GE8Hsi+FL16G3gngvPEsR3q4ADneLtalMZOZq8CmVR7TIRdeA='} 2024-04-22T06:38:15.323 DEBUG:teuthology.nuke:shortname: smithi070 2024-04-22T06:38:15.323 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-22T06:38:15.363 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi070.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668133', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-22 06:18:48.196392', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEcso0amEbD7LWkysbF+kOSM3IVibZdDd54bL/6XuJSZ7MZk97LGx/Xh4KeuLIRyxLqN83Xjbno3yb2y3nWBpwk='} 2024-04-22T06:38:15.410 INFO:teuthology.orchestra.console.smithi044:Power off 2024-04-22T06:38:15.410 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-22T06:38:15.440 INFO:teuthology.orchestra.console.smithi070:Power off 2024-04-22T06:38:15.440 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-22T06:38:15.449 DEBUG:teuthology.orchestra.console.smithi044:power off output: Chassis Power Control: Down/Off 2024-04-22T06:38:15.450 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T06:38:15.471 DEBUG:teuthology.orchestra.console.smithi070:power off output: Chassis Power Control: Down/Off 2024-04-22T06:38:15.471 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T06:38:15.484 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on 2024-04-22T06:38:15.503 DEBUG:teuthology.orchestra.console.smithi070:check power output: Chassis Power is on 2024-04-22T06:38:19.485 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T06:38:19.504 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T06:38:19.596 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on 2024-04-22T06:38:19.614 DEBUG:teuthology.orchestra.console.smithi070:check power output: Chassis Power is on 2024-04-22T06:38:23.598 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T06:38:23.615 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T06:38:23.708 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is off 2024-04-22T06:38:23.725 DEBUG:teuthology.orchestra.console.smithi070:check power output: Chassis Power is off 2024-04-22T06:38:23.809 INFO:teuthology.orchestra.console.smithi044:Power off completed 2024-04-22T06:38:23.826 INFO:teuthology.orchestra.console.smithi070:Power off completed 2024-04-22T06:38:23.931 INFO:teuthology.run:Summary data: description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/with-quiesce 2-workunit/fs/trivial_sync}} duration: 793.3153553009033 failure_reason: 'Command failed on smithi044 with status 124: ''sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json''' flavor: default owner: scheduled_leonidus@teuthology status: fail success: false 2024-04-22T06:38:23.931 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-22T06:38:24.011 INFO:teuthology.run:FAIL